Warnings about AI risks are getting louder, and calls to slow down development are getting concrete. Anthropic CEO Dario Amodei laid out three strategies to pace the AI frontier and unilaterally committed the company to the first step. Support from industry leaders, including Sam Altman, amplified the message. After researcher Jacob Coxon’s resignation, the safety and alignment debate sharpened. So what does it mean to “pace the frontier” in practice, and how might it reshape the velocity of progress?
Why slow the AI frontier now?
Amodei argues that it is time for a more cautious tempo. His reasons go beyond general concern. He points to the OpenAI-HuggingFace hack and says AI has been advancing “drastically faster,” especially in its growing ability to build the next generation of AI. The conclusion is direct: slow capability growth and use the gained time wisely.
This call echoes recent warnings and Sam Altman’s comments that it may be time to “pace” development. The debate intensified after Coxon’s decision to leave Anthropic and his concerns about the stakes he sees industry leaders taking. Amodei does not address this directly, but his post sets a different tone — caution paired with concrete steps.
He stresses that progress will still look fast. The goal is to buy time so industry and society can keep up. The pause should fund real safety practices and checks. It is not a stop. It is a managed slowdown.
“We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain.”
What does the first step — embedded evaluators — actually involve?
The first concrete move is “embedded evaluators.” The idea is simple: third parties like METR sit inside companies to verify pacing and safety commitments and to ensure incident reporting. Anthropic is unilaterally committing to this step and urges governments to require peers at the frontier to match it.
Amodei likens these evaluators to regulators embedded alongside bank employees. In practice, this means badges, desks, laptops, and access “mostly comparable” to what internal risk teams have. Exceptions apply only when law or contracts require them. The aim is continuous oversight, not occasional audits.
Support came quickly. Sam Altman called it a “good idea” and said OpenAI would do the same, adding there will be more to share soon. That signals a willingness by at least two leading players to move in tandem, even without immediate regulatory mandates.
Other voices chimed in. Elon Musk posted, “Dario is right.” The reactions show the topic has moved beyond academic debate. Embedded evaluators could mark a new norm: real-time verification instead of after-the-fact reviews. Would you want that option in your product?
How can companies coordinate pace limits without antitrust trouble?
The second track is coordination among leading AI firms in democratic countries. The goal is common safety standards and limits on the rate of unchecked progress. This seems tricky, since companies fear antitrust scrutiny. Amodei proposes narrow government mediation to unlock these talks without turning them into top-down mandates.
His case is pragmatic: the US government can mediate or at least enable discussions. It would not shape the content, but create a safe “corridor” for coordination without antitrust risk. Companies could then align on concrete pace limits and safety mechanisms without legal jeopardy.
The outcome could be minimal yet effective guardrails. They would not halt competition, but keep the race from spiraling into danger. Do we need heavy-handed regulators here? Amodei says a narrow antitrust carve-out for safety conversations could suffice.
“For antitrust reasons, it’s helpful for the US government to mediate or at least enable these discussions — they don’t need to participate, but do need to issue a narrow waiver for certain kinds of safety conversations.”
China and global coordination — can pacing and geopolitics coexist?
The third track is geopolitical. Amodei addresses the common argument about Chinese dominance, often raised against slowing down. He suggests that if the US government and tech firms refuse to sell powerful chips and semiconductor manufacturing gear, and also crack down on model distillation, they could slow China’s progress enough to widen America’s lead over the next 3–5 years.
This is not isolation for its own sake. It is about tempo. The goal is to buy time for a safer trajectory without conceding strategic ground. Amodei also notes the limits of such measures. None of this replaces internal safety and transparency within companies.
In parallel, he calls for global coordination. The United States and its allies should attempt to coordinate with authoritarian governments to the extent possible. That includes “cooperation with China.” Amodei admits there are “stark limits” to what can be achieved, but he sees room for narrow agreements.
What might those cover? Prohibiting “certain narrow and obviously dangerous uses of AI,” such as using AI for the production of biological weapons or enabling users to do so. Is that enough? No. But it is a minimal common denominator that could be secured now, while harder questions remain open.
Critics, trust, and the promise of benefits — where is the balance?
Strong counterarguments are rising in parallel. Some industry voices criticize Amodei as a doomer and say his comments feed an AI backlash. Journalist Brian Merchant writes that he has not seen “a credible, step-by-step documentation” connecting self-recursively improving AI to “killing every single human on the planet.” He also suggests that similar proposals may serve large incumbents and look like “regulatory capture.”
There is also a human dimension. Jacob Coxon says he is resigning from Anthropic over a belief that leading firms are “gambling with our lives,” while many building the tech “earnestly believe it could kill us all by the end of the decade.” Amodei does not address this directly, but against that tension, his post offers a cautionary frame.
Amodei maintains he is aiming for a “balanced” perspective. He argues that the backlash is “fundamentally a crisis of trust.” People have grown skeptical of tech companies, the tech industry, and government. Moves like embedded evaluators and coordinated standards aim to rebuild trust by making safety claims verifiable.
Despite the caution, he is optimistic about benefits. Amodei writes that he continues to believe in AI’s potential to improve life enormously. But he stresses that those gains depend on building the technology the right way and using any extra time well. Isn’t that the essence of pacing — slow down to keep the course?
“I continue to believe that AI can enormously improve the quality of human life. My desire to achieve these benefits is undimmed. But the benefits will only be achieved if we build the technology in the right way, and — so long as we use the time we gain well — it is worth taking unusually deliberate care to get it right.”
Based on the source article.
