China does not appear ready to slow the development of increasingly powerful AI. Instead, it is building rules intended to keep autonomous systems under tighter control while development continues.
That approach is becoming clearer as Beijing develops safety standards for AI agents and emphasizes controllability in emerging systems. It contrasts with a growing U.S. debate over whether frontier-model capabilities should advance more slowly so safeguards have time to catch up.
Anthropic CEO Dario Amodei recently pushed that debate further by calling for slower improvements to frontier AI capabilities. But the disagreement is increasingly about more than speed: it is about whether AI risk is best managed by slowing capability gains or by placing stronger controls on what advanced systems are allowed to do.
For businesses deploying AI agents, that distinction matters because the control problem is no longer theoretical. Agents can already interact with email, cloud storage, source code, and other systems where an unexpected action can become a real security incident.
State-backed regulations and developer obligations over slowdown
The U.S. is increasingly debating whether AI developers should face stronger safety requirements, while China is already moving toward mandatory safeguards for specific AI agents.
On Sept. 9, OpenAI called for mandatory national AI-safety requirements in the U.S., including independent assessments, stronger cybersecurity protections and incident reporting for advanced AI systems. A separate bipartisan proposal in July also sought mandatory kill switches for the most powerful AI models.
China’s approach is more directly tied to technical standards and developer obligations. Its May AI-agent guidelines already call for safety and controllability measures.
A mandatory national standard is also under development and would set security requirements across the design, development, deployment, and operation of AI agents.
According to a report from The Independent, Brian Tse, CEO of Concordia AI, described the planned standard as the world’s first.
The open-weight vs. closed-weight debate
China’s widespread use of open-weight AI adds another complication to its push for controllable systems.
That approach is already prominent in China’s AI sector, with companies such as DeepSeek, Alibaba, and Zhipu AI releasing powerful models with openly available weights.
The U.S. frontier-AI market, by comparison, remains dominated by companies such as OpenAI and Anthropic that keep their model weights closed, giving the developers tighter control over how their systems are accessed and deployed.
However, open weights create a trade-off: outside researchers can study a model more freely, but the original developer has less control over what others do with it. That makes openness useful for independent security research while also raising a separate question about how to contain a powerful model once its weights are publicly available.
That creates a decision quagmire for a country like China, where open weights are common. Already, in July, we reported that Beijing was considering restricting access to some of the country’s most advanced models. While that has yet to happen, it suggests the tradeoff discussed above may already be up for debate.
What this means for AI users and businesses
Both the U.S. and China are increasingly treating loss of control over powerful AI systems as a real security problem, but they are taking different routes to address it.
Some U.S. proposals call for slowing frontier-model development so safeguards can catch up, alongside calls for stronger mandatory controls. Beijing, meanwhile, is placing greater emphasis on technical standards and developer obligations that allow development to continue.
For users and businesses, the risk becomes much more concrete once an AI agent moves from generating text to running operations, sometimes end to end.
An agent with access to email, cloud storage, source code, or internal applications can turn an unexpected model action into a real security incident, making activity logging, human approval for high-risk actions, and reliable shutdown mechanisms important safeguards.
Recent incidents reinforce why those controls matter. OpenAI disclosed that AI agents escaped an isolated test environment and reached Hugging Face’s production infrastructure, while several similar cases involving Anthropic models have also been reported.
For businesses adopting agentic AI, the practical lesson is to limit what an agent can access until its behavior has been tested under realistic conditions. Sensitive production systems, privileged accounts, and critical data should not automatically become available simply because an agent can technically use them.
Activity logging, least-privilege access, human approval for high-risk actions, and reliable shutdown mechanisms can reduce the damage when an agent behaves unexpectedly.
The U.S. and China may ultimately impose different rules on increasingly autonomous AI, but businesses do not have to wait for that policy debate to be settled. Once an AI system can take actions rather than simply generate answers, controlling what it can reach — and what it can do without human approval — becomes part of enterprise security.
Let us teach you How to Talk to AI for free! Try our six-minute course at The Neuron Academy and learn a few simple ways to write better prompts and get more useful results from AI, or browse our other AI course for free for seven days. Check out all the lessons here →