Data poisoning poses a significant threat to AI models, compromising their integrity and reliability. As organisations increasingly rely on AI technologies, safeguarding AI models from data poisoning attacks becomes paramount.

In this post, we explore the role of the law in protecting AI models from data poisoning and supplement it with technical approaches that can further enhance their defence against this malicious practice.

Understanding data poisoning in AI

What is it?

Data poisoning in AI involves manipulating or contaminating training data to deceive or compromise the performance of AI models. Typically, bad actors inject malicious or misleading data, aiming to introduce bias, disrupt decision-making, or cause harmful actions based on manipulated information.

A practical example in the financial services industry

Consider a scenario where a bad actor aims to manipulate the credit scoring process of a bank’s AI model. They inject manipulated data into the training dataset, targeting factors such as income, employment history, or credit history. The injected data is designed to misrepresent the creditworthiness of certain individuals.

As a result, the AI model, influenced by the poisoned data, generates inaccurate credit scores for affected individuals. This can lead to biased lending decisions, with deserving applicants being denied loans or offered unfavourable terms while undeserving applicants gain access to credit they should not have received.

This example highlights how data poisoning can exploit AI models used by banks, distorting credit scoring systems and potentially causing financial harm to both individuals and the institution. It underscores the importance of protecting AI models against data poisoning to maintain fairness and trust in financial services.

Protecting AI models from these attacks is essential to ensure the reliability and fairness of AI systems. But how can the law help you do so?

Legal measures to combat data poisoning

Data protection law

Data protection laws, like POPIA and the GDPR, play a crucial role in safeguarding AI models from data poisoning attacks. These laws outline the responsibilities of controllers and processors, emphasising the importance of data security, lawful bases, and data anonymisation. Ultimately, organisations must comply with these laws to protect AI models and the privacy of individuals’ data.

Intellectual property and trade secret laws

Intellectual property laws can offer protection against unauthorised access or misuse of proprietary data used in AI models.

It would be best to take legal measures, including trade secret protection and non-disclosure agreements, to safeguard your valuable datasets from data poisoning attempts. These laws can deter bad actors and provide legal remedies in case of data poisoning incidents.

Technical approaches to protect AI models

  1. Data pre-processing and anomaly detection: Robust data pre-processing techniques, including data cleaning, normalisation, and feature engineering, help detect and mitigate data poisoning attempts. Further, anomaly detection algorithms can identify suspicious patterns or outliers in the training data, providing additional protection against malicious data injection.
  2. Model robustness and monitoring: Building AI models with robustness in mind can enhance their resistance to data poisoning attacks. Techniques such as adversarial training, where models are trained on both clean and manipulated data, can increase their resilience. Ongoing model monitoring and validation can help identify anomalies, drifts, or biases that may indicate data poisoning attempts.
  3. Explainability and interpretability: Implementing explainable AI techniques helps detect data poisoning by providing insights into how AI models arrive at their decisions. Understanding the model’s decision-making process makes identifying potential biases or anomalies caused by poisoned data easier.

Actions to take next

  • Plan for data poisoning incidents effectively by asking us to draft or review your AI incident response plan.
  • Know how to respond to data poisoning incidents by asking us to workshop the topic with your team.
  • Respond to AI incidents promptly by reaching out to us for incident response coaching.
  • Recover from AI incidents safely by asking us for a framework for incident recovery.