AI is advancing faster than lawmakers can react — and nowhere is that tension more visible than in New York.
In June 2025, state lawmakers passed the Responsible AI Safety and Education (RAISE) Act. This first-of-its-kind proposal would require the largest AI developers to follow basic safety rules. The bill has cleared the Legislature but still awaits the governor’s signature, leaving New York poised — but not yet committed — to becoming the first state to directly regulate frontier AI.
Supporters argue the measure is a necessary guardrail as AI approaches capabilities once confined to science fiction. Critics warn it could slow innovation and fracture national policy. Either way, the RAISE Act has become a flashpoint in a rapidly escalating fight over how to keep powerful AI systems in check.
The bill’s focus: Limiting ‘critical harm’
The RAISE Act would require large AI companies to publish safety and risk protocols and disclose serious safety incidents.
The bill takes a targeted approach, focusing its requirements only on the “largest AI companies” — those that have spent over $100 million in computational resources to train advanced AI models (known as “frontier models”). Its core purpose is to prevent “critical harm,” which the bill defines as an event causing the death or serious injury of 100 or more people or at least $1 billion in property damage.
This focus is directed at damage caused or materially enabled by specific catastrophic scenarios. These include the creation or use of a chemical, biological, radiological, or nuclear weapon, or an AI model autonomously engaging in criminal conduct without meaningful human intervention.
For these large developers, the law mandates several core safeguards. Companies must first develop and implement a written “Safety and Security Protocol” detailing how they will reduce the risk of critical harm. Crucially, developers are prohibited from deploying a frontier model if doing so would create an unreasonable risk of critical harm.
To ensure transparency and accountability, a version of this protocol with appropriate redactions to protect trade secrets or public safety must be conspicuously published and sent to the New York State Attorney General and the Division of Homeland Security and Emergency Services. The full, unredacted protocol is accessible to the Attorney General upon request.
Furthermore, companies must disclose a “Safety Incident” (a known critical harm or incident that demonstrates an increased essential risk of harm) to the Attorney General and Homeland Security within 72 hours of learning of it.
Suppose a company fails to meet these standards. In that case, the New York State Attorney General is empowered to impose substantial civil penalties: up to $10 million for a first violation and up to $30 million for subsequent violations.
Why the urgent need for regulation?
Supporters argue that AI is rapidly advancing, bringing both immense benefits and catastrophic risks. Assemblymember Alex Bores, a co-sponsor, highlighted the dilemma, stating, “But the same pathways that will allow it to potentially cure diseases [will] allow it to, say, build a bio weapon,” per CNBC.
Experts also echo this concern. The International AI Safety Report, which includes over 100 experts from 30 countries, warned of potential risks including “AI-enabled hacking or biological attacks.” Specific reports from the AI community also fuel the sense of urgency.
For instance, OpenAI noted that its models are “on the cusp” of being able to help a novice create a known biological threat. Autonomy is another major concern, with one report noting that “AI models have been observed preventing themselves from being shut down despite explicit instructions to the contrary.”
State Senator Andrew Gounardes, the bill’s sponsor, summed up the need for action by drawing an analogy: “Would we let automakers sell a car with no brakes? Of course not. So why would we let developers release incredibly powerful AI tools without basic safeguards in place?”
“New Yorkers want technology to make their lives better and easier, not put their health and safety at risk. My RAISE Act ensures AI can flourish while requiring the largest companies to have a safety plan so their products aren’t used to hurt people,” Senator Gounardes added. “It’s exactly the type of reasonable, common-sense safeguard we’d expect of any company working on a potentially dangerous product, and it ensures no one has an incentive to cut corners or put profits over safety.”
More must-read AI coverage
- SS&C Intralinks DealCentre AI vs. Datasite: Which platform is built for the future of dealmaking?
- SS&C Intralinks FundCentre AI vs. Juniper Square: Which platform better supports modern private markets fund managers?
- Why Data, Not Models, Determines AI Success
- The Rise of the AI-Native Factory: How Physical AI Is Transforming Manufacturing
Facing industry backlash
While the bill has strong support among New Yorkers, it faces stiff opposition from powerful players in the tech world.
A bipartisan Super PAC called Leading the Future (LTF), backed by high-profile figures including OpenAI President Greg Brockman and Palantir co-founder Joe Lonsdale, has launched a campaign specifically targeting Assemblymember Bores, who is running for a Congressional seat.
The group and its leaders oppose state-level regulation, arguing it will “stifle innovation” and create a “patchwork, uninformed, and bureaucratic state laws that would slow American progress.” They believe this would open “the door for China to win the global race for AI leadership.”
Assemblymember Bores has countered this pushback, telling CNBC’s Squawk Box, “They don’t want there to be any regulation whatsoever.” He stressed that the bill simply ensures that developers who have made voluntary safety commitments keep their promises, noting that “all the RAISE Act does is put those commitments into law.”
New York is positioning itself to be a national leader in AI regulation, acting on the recommendation of companies like Anthropic, which warned that the federal process would not be fast enough to address current risks. The RAISE Act focuses on a narrow set of severe risks, allowing other important issues, such as bias and workforce impact, to be addressed in future legislation.
The law is set to take effect 90 days after it is signed. The final decision now rests with Governor Hochul, who has until the start of the 2026 session to sign the bill into law.
OpenAI’s warning that the country needs far more construction workers and electricians to build out data centers underscores how physical infrastructure is becoming a key constraint alongside new safety rules.