# What is Model Poisoning?

> Model poisoning is a training-stage attack in which an adversary manipulates a model or its parameters to change later behavior.

- Canonical URL: https://yellowcube.eu/glossary/model-poisoning/
- Publisher: Yellow Cube
- Language: en
- Contact: hello@yellowcube.eu

## Content

Rather than corrupting learning examples, the attacker directly influences weights, gradients, updates, checkpoints or a model artifact. The result may be broad performance degradation, targeted errors, an attacker-chosen backdoor or a loss of model integrity that appears only under particular conditions.

Relevant attack paths include a compromised participant submitting malicious updates during federated learning, unauthorized changes inside a training pipeline, and a poisoned pretrained or fine-tuned model introduced through the AI supply chain. Protection therefore depends on knowing where model artifacts and updates originated, who could modify them and what evidence supports promotion into production.

### Key points

- **Attack surface:** Review model registries, artifact repositories, training jobs, distributed or federated participants, update aggregation, serialization formats, deployment pipelines and third-party model sources.
- **Integrity controls:** Restrict write access, separate duties, sign and verify artifacts where supported, retain lineage and hashes, approve trusted sources and protect the systems that build and distribute models.
- **Evaluation:** Compare candidate behavior with independently protected baselines and holdouts, test for targeted triggers and unexpected capability changes, and investigate unexplained differences before release.
- **Response:** Preserve the suspect artifact and provenance evidence, remove affected versions from distribution, identify downstream deployments, restore a trusted version and address the compromised update path.
- **Important limitation:** Model evaluation cannot exhaust every possible input or hidden behavior. Passing a benchmark does not prove parameter integrity, while an ordinary authorized fine-tune or an accidental training defect is not poisoning unless adversarial manipulation is involved.

### Related terms

[Data poisoning](<https://yellowcube.eu/glossary/data-poisoning/>) · [Adversarial machine learning](<https://yellowcube.eu/glossary/adversarial-machine-learning/>) · [Software supply-chain security](<https://yellowcube.eu/glossary/software-supply-chain-security/>) · [AI security posture management (AI-SPM)](<https://yellowcube.eu/glossary/ai-security-posture-management/>) · [AI security](<https://yellowcube.eu/glossary/ai-security/>)

### Sources

[NIST AI 100-2 E2025, Adversarial Machine Learning Taxonomy](https://csrc.nist.gov/pubs/ai/100/2/e2025/final) · [MITRE ATLAS](https://atlas.mitre.org/) · [CISA and UK NCSC Guidelines for Secure AI System Development](https://www.cisa.gov/news-events/alerts/2023/11/26/cisa-and-uk-ncsc-unveil-joint-guidelines-secure-ai-system-development)

## Attribution and scope

This Markdown representation is generated from the same approved content records as the canonical HTML page. Cite the canonical URL above when referencing this material. Product and service descriptions are informational; confirm project-specific requirements with Yellow Cube.