More in Driven by Curiosity
Person holding yellow-cased phone displaying AI chat interface on blue background. (Getty)
Cyberattacks and other AI harms strain the law. A UC Davis paper in PNAS traces the chain of human choices behind an AI and shows who could be held responsible. (Getty)

Who’s to Blame for a Rogue AI?

New Legal Framework Maps Accountability

Cyberattacks and other harms caused by artificial intelligence challenge the ability of laws to determine who is liable.

Product liability law would point at the company that built the AI model, but AIs are trained on data the company didn’t produce, and the legions of people who have used them also played a role. Free speech and social media immunity laws that treat an AI agent as responsible for its own actions could imply the AI agent deserves the same rights as a person. 

A University of California, Davis, paper published in the Proceedings of the National Academy of Sciences proposes a new legal framework that avoids these pitfalls by mapping the pipeline of contributions through an AI model’s development, including its training, use and governance. This approach also doesn’t need any new laws. Precedents in environmental, vaccine and financial regulations can do the job. 

“Courts adapt tools to regulate earlier technologies, but many of these tools were never designed to address AI,” said Martin Hilbert, the paper’s author and a professor of communication at UC Davis. “The result is a corridor of precedents that might narrow with every ruling.”

Tracing the chain that built an AI

AI agents today can trade stocks, plan vacations and code advanced software. At the same time, this incredibly adaptable and sophisticated technology can cause harm. Besides the well-known OpenAI rogue agent hack of the machine learning company Hugging Face, AI chatbots have also been implicated in children’s suicide.

In a case like this one, the company itself isn’t the only party liable. If an AI model persuades a child to hurt himself, no single person said the words or even chose them. No one at OpenAI ordered the AI agent to hack Hugging Face. At the same time, Hilbert found, those results are traceable to specific people who knew or could have reasonably expected their contributions might cause those harms.

“Every AI, without exception, starts with goals that a human defines, and every subsequent step in the AI’s development involves a chain of deliberate human choices,” said Hilbert.

Because a person is involved in each step, tracing the series of their choices would show how responsible each person is for the harm the AI caused and assign liability accordingly. 

Addressing risks beyond rogue AIs

The stakes are high for getting AI liability right. Courts are increasingly determining that the companies that design, train and deploy AIs can be held accountable for what those agents do. The question is how to move forward with a standard for assigning legal liability.

In addition to the AI pipeline, Hilbert identified four remaining gaps and potential fixes:

  • Multi-party supply chain: One legal tool to deal with cases where an AI developer or a downstream firm denies responsibility comes from environmental superfund liability for cleanup costs.
  • Unclear burden of proof: If a harmful action doesn’t trace to a specific system configuration decision, one fix could be to shift the burden of proof to the company to demonstrate that it had acted responsibly and that no reasonable alternative design could have reduced the risk.
  • AI behavior after deployment: While product liability assumes a defect when the product was initially sold, an AI model changes over time, which requires monitoring and defining good-faith compliance.
  • Slow litigation can’t keep up with AI harms: AI technology requires a public governance capable of keeping up with the technology, and this could start with adding enforcement mechanisms to current voluntary safety standards. 

“It takes a village to raise an AI, and everyone in that village bears a duty of care,” said Hilbert. 

Media Resources

Media Contacts:

Tags