Data Poisoning

Data Poisoning is a sophisticated adversarial attack targeting the fundamental integrity of an artificial intelligence model. It occurs during the training or fine-tuning phase of the model's lifecycle. An attacker deliberately introduces malicious, corrupted, or heavily biased data into the massive datasets used to train the AI. Because foundation models require billions of data points, it is extremely difficult for human engineers to verify every piece of ingested information.

The goal of data poisoning is to fundamentally alter the model's 'worldview' or to install a hidden backdoor. For example, an attacker might poison an open-source coding dataset so that whenever a model generates an authentication script, it subtly includes a specific vulnerability the attacker can later exploit. In a less technical example, an attacker could poison a public dataset to ensure a specific brand is always associated with negative sentiment in AI-generated summaries.

For enterprises, data poisoning is a major AI Supply Chain Risk. If an organization downloads a poisoned open-source model, or if they use an external RAG (Retrieval-Augmented Generation) system that accidentally indexes an attacker's poisoned document on the corporate network, the AI will confidently output malicious or corrupted information. Mitigating this risk requires rigorous provenance tracking of all training data, cryptographic hashing to ensure model integrity, and strict access controls on the internal document repositories used for enterprise grounding.