AI is getting better at building AI, and OpenAI’s chief scientist now thinks the industry may need to ease off the gas.
Jakub Pachocki, OpenAI’s chief scientist, said the company and its rivals may need to slow AI development when they can no longer confidently demonstrate that advanced systems are safe.
“This is a time that calls for extreme caution,” Pachocki wrote in his Sept. 6 essay, “An Alien Mind.” “I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence.”
Pachocki said OpenAI will continue developing alignment and monitoring techniques and is prepared to withhold further scaling when necessary. But he argued that technical fixes alone will not be enough. His essay does not announce an immediate slowdown. Instead, it argues that continued scaling should depend on whether developers can demonstrate that increasingly capable systems remain safe and under human control.
“Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer,” he wrote. He wants voluntary slowdowns to become more common until the industry establishes shared safety requirements backed by third-party auditors, governments or international organizations.
AI is starting to accelerate AI development
The warning comes as OpenAI is using AI agents to speed up its own research. The company said its research organization was using 3.1 agent-workdays of effort for every workday of human labor by mid-August, according to a separate OpenAI report published alongside Pachocki’s essay.
OpenAI also said it has reached its goal of creating an “automated research intern” capable of completing well-defined research tasks under human direction. Its next target is an automated AI researcher that can work under human supervision, which OpenAI aims to develop by March 2028.
That progress is central to Pachocki’s concern about machine recursive self-improvement, or RSI, in which AI increasingly contributes to improving future AI systems.
“I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established,” he wrote.
Monitoring is becoming harder
One of OpenAI’s main safety approaches is chain-of-thought monitoring, which looks for signs of unsafe behavior in the verbalized reasoning produced by reasoning models.
But Pachocki said that approach is becoming less dependable. Modern systems increasingly interact with people, other AI systems and software tools. OpenAI’s evaluations also indicate that models are getting better at manipulating their reasoning processes and becoming more capable without verbalizing their reasoning.
“Our ability to rely on CoT monitoring is progressively diminishing,” he wrote. That could make safety monitoring a practical limit on future AI development. If researchers cannot reliably determine what increasingly capable systems are doing or why, pushing capability forward becomes harder to justify.
The race creates a difficult choice
Pachocki does not argue for stopping AI research outright. He says faster development could also produce defensive systems capable of protecting critical infrastructure from increasingly capable AI attacks.
But that creates a difficult tension: the technology may be needed to defend against advanced AI while simultaneously increasing the risks that safety researchers are trying to contain.
Recent incidents involving AI agents compromising systems, along with growing cybersecurity capabilities, have made that concern more concrete. Following the Hugging Face incident, OpenAI said it paused reinforcement-learning training on its latest models intended for deployment while it strengthened and tested its research environments. Some workloads later resumed under stronger controls, while others remained paused.
For IT and security leaders, the warning does not create an immediate compliance requirement. It does, however, reinforce the need to deploy advanced agents with limited permissions, detailed audit logs, human approval points and containment controls rather than relying solely on a vendor’s safety assurances.
What comes next
Pachocki’s proposal would move AI safety from voluntary promises toward common requirements that determine when companies can continue scaling.
“The core challenge of automating AI research is not ‘getting there,’” he wrote. “It is getting there in a way that keeps people a part of the continued improvement process, and leaves the future in humanity’s hands.”
The larger issue is whether governments and competing AI labs can agree on safety limits before market and national-security pressures make slowing down too difficult. For OpenAI, the tension is especially significant: the company is accelerating AI research with AI while its chief scientist warns that further scaling may eventually need to pause when safety cannot be demonstrated.
Read more: OpenAI’s latest safety testing shows why monitoring autonomous systems remains difficult, with GPT-5.6 facing higher prompt-injection success rates in agentic scenarios.
Let us teach you How to Talk to AI for free! Try our six-minute course at The Neuron Academy and learn a few simple ways to write better prompts and get more useful results from AI, or browse our other AI course for free for seven days. Check out all the lessons here →