[A] vision for red-teaming in the public interest: a process that goes beyond system-centric testing of already built systems to consider the full range of ways the public can be involved in evaluating genAI harms

Report by Ranjit Singh Borhane Blili-Hamelin Carol Anderson Emnet Tafesse Briana Vecchione Beth Duckles and Jacob Metcalf. The increasing power and availability of generative AI (genAI) systems has led regulators, technologists, and members of the public to call for new safety practices to anticipate harms and protect the public interest. One early and promising approach, drawing from cybersecurity and military practices, is red-teaming, in which designated teams use adversarial methods to identify vulnerabilities in systems. Drawing on 26 semi-structured interviews and participant observation at three public red-teaming events — and based on a collaborative research project between Data & Society and AI Risk and Vulnerability Alliance (ARVA) — Red-Teaming in the Public Interest examines how red-teaming methods are being adapted to evaluate genAI.

Red-teaming genAI raises not only methodological questions — how and when to red-team, who should participate, how results should be used — but also thorny conceptual questions: whose interests are being protected? What counts as problematic model behavior, and who gets to define it? Is the public an object being secured, or a resource being used? In this report, Ranjit Singh, Borhane Blili-Hamelin, Carol Anderson, Emnet Tafesse, Briana Vecchione, Beth Duckles, and Jacob Metcalf offer a vision for red-teaming in the public interest: a process that goes beyond system-centric testing of already built systems to consider the full range of ways the public can be involved in evaluating genAI harms.