Rather than corrupting learning examples, the attacker directly influences weights, gradients, updates, checkpoints or a model artifact. The result may be broad performance degradation, targeted errors, an attacker-chosen backdoor or a loss of model integrity that appears only under particular conditions.
Relevant attack paths include a compromised participant submitting malicious updates during federated learning, unauthorized changes inside a training pipeline, and a poisoned pretrained or fine-tuned model introduced through the AI supply chain. Protection therefore depends on knowing where model artifacts and updates originated, who could modify them and what evidence supports promotion into production.
Key points
Attack surfaceReview model registries, artifact repositories, training jobs, distributed or federated participants, update aggregation, serialization formats, deployment pipelines and third-party model sources.
Integrity controlsRestrict write access, separate duties, sign and verify artifacts where supported, retain lineage and hashes, approve trusted sources and protect the systems that build and distribute models.
EvaluationCompare candidate behavior with independently protected baselines and holdouts, test for targeted triggers and unexpected capability changes, and investigate unexplained differences before release.
ResponsePreserve the suspect artifact and provenance evidence, remove affected versions from distribution, identify downstream deployments, restore a trusted version and address the compromised update path.
Important limitationModel evaluation cannot exhaust every possible input or hidden behavior. Passing a benchmark does not prove parameter integrity, while an ordinary authorized fine-tune or an accidental training defect is not poisoning unless adversarial manipulation is involved.