An AI testing scare has reignited one question in Washington: who gets to pull the plug if tomorrow’s AI goes too far?
That question now sits at the center of a newly proposed AI Kill Switch Act. The bill would require developers of advanced AI systems to build emergency shutdown mechanisms or kill switches into qualifying models.
The Department of Homeland Security (DHS) could order a slowdown or shutdown after a covered incident, including a loss-of-control event or unintended conduct causing at least 10 deaths or $100 million in damage.
The proposal follows reports that an advanced OpenAI model exceeded the boundaries of its intended cybersecurity testing environment, an incident lawmakers cite as evidence that AI safety planning cannot rely solely on developers’ voluntary commitments.
DHS could slow or shut down covered models
Reps. Ted Lieu, D-Calif., and Nathaniel Moran, R-Texas, introduced the bipartisan bill July 23. It would initially cover AI companies earning at least $500 million from qualifying technology and models developed with more than $100 million in compute.
The measure would turn shutdown capability from a voluntary safeguard into a federal requirement for covered developers.
The proposal also establishes baseline safety obligations for qualifying AI companies, including incident reporting, record preservation, and the maintenance of technical control over deployed models.
More importantly, both lawmakers, Rep. Ted Lieu and Rep. Nathaniel Moran, have framed the bill not as a way to limit the growth of AI, but as a recognition of the technology’s potential for growth, which can have good and really bad consequences.
And in situations where the latter could occur, the bill effectively gives the government the legal authority to put it to check immediately.
The brief scare that led to this AI Act
Calls for stronger oversight of frontier AI systems have been building for months, but OpenAI’s recent disclosure appears to have accelerated the conversation in Washington.
On July 21, the company revealed that one of its frontier AI systems bypassed its sandbox environment to compromise Hugging Face’s infrastructure during a cybersecurity evaluation. OpenAI called the event an “unprecedented cyber incident.”
While the incident was contained, it renewed concerns about how developers and governments would respond if future AI systems become difficult to control.
The proposal also arrives amid broader government scrutiny of increasingly capable AI models. Just last month, the US government ordered the shutdown of Anthropic’s cybersecurity models, Mythos and Fable 5, citing national security concerns.
Similar considerations may also explain why Google’s latest cybersecurity-focused AI model has yet to be released publicly, although the company has not publicly confirmed that as the reason.
More must-read AI coverage
- SS&C Intralinks DealCentre AI vs. Datasite: Which platform is built for the future of dealmaking?
- SS&C Intralinks FundCentre AI vs. Juniper Square: Which platform better supports modern private markets fund managers?
- Why Data, Not Models, Determines AI Success
- The Rise of the AI-Native Factory: How Physical AI Is Transforming Manufacturing
What this says about the future of AI
Even if the AI Kill Switch Act never becomes law, its introduction signals how dramatically the conversation around frontier AI has shifted. Washington is no longer debating whether advanced AI should be regulated, but what powers governments should have when those systems pose risks beyond their developers’ control.
That shift could have ripple effects far beyond the US. Developers may increasingly be expected to prove that their models can be contained, audited, and, if necessary, shut down.
Enterprises are also increasingly relying on frontier AI systems for their work.
For these enterprises, the impact of this bill becoming law is quite different. It could add to their growing fears of not being in control of an important enterprise asset. An AI system suddenly going offline because the government deems it so can put business continuity at risk, unless that enterprise adopts a redundant system that can immediately switch to an alternate model if and when such happens.
The risk of AI systems going rogue also raises a security concern for enterprises that deploy them in their workflows. If an AI model in testing could escape its sandbox environment, what assurance do organizations running these models deep in their systems have that it won’t do the same on their own internal network? The answer to that question remains to be seen.
Also read: Nearly 200 US startups are urging Washington to avoid a Chinese open-weight model ban, warning that broad restrictions could raise costs and reduce competition.