The learning process absorbs the manipulation, which can reduce overall performance, bias a targeted decision or create a backdoor that activates when a chosen trigger appears. The attacker may compromise a data source, collection pipeline, annotation process, feedback loop or dataset repository.
The term primarily concerns data that affects learning, including pretraining, fine-tuning and reinforcement or feedback datasets. Manipulating a retrieval source at inference time is usually described more precisely as retrieval-augmented generation poisoning or knowledge-base poisoning; corrupting an evaluation set can invalidate assurance even if it does not alter the deployed model.
Key points
Data lineageRecord origin, collection method, license, transformations, labels, versions, approvals and the models or experiments that consumed each dataset.
Preventive controlsLimit and audit write access, separate trusted evaluation data, authenticate approved revisions, protect pipelines and repositories, and assess externally sourced or user-contributed data.
DetectionUse statistical and semantic review, duplicate and outlier analysis, targeted sampling, source comparisons and tests for suspicious triggers or concentrated influence.
RecoveryPreserve evidence, identify affected data and derived models, revert to a trusted dataset version, retrain where necessary and close the ingestion or authorization weakness.
Important limitationPoisoned examples may look legitimate and a small, carefully chosen subset can be influential. Data cleaning reduces exposure but cannot certify that a dataset is benign; poor-quality or unrepresentative data without adversarial intent is a data-quality problem, not poisoning.