The AI industry’s loud debate just got louder. Researcher Jacob Coxon said he resigned from Anthropic, arguing leading companies are “gambling with our lives.” Then Anthropic’s alignment lead posted, “We really do earnestly believe AI could kill all humans!” and added a personal estimate of “>10% within the next decade.” The Equity podcast hosts examined the apocalyptic notes, the skepticism, and the potential PR effect. They recorded the episode before Anthropic’s CEO Dario Amodei published a plan for more cautious development.
Why did the debate ignite now?
It accelerated after a public resignation and immediate amplification on social media. Coxon claimed leading companies are “gambling with our lives,” then left Anthropic. The company’s alignment lead reinforced the wave, repeating that AI could kill all humans, with odds above 10% in the next decade. Such framing sharply reset the conversation and poured fuel on an already hot debate.
Why now? Three forces collided: a stark statement from a young researcher, instant endorsement by a high‑profile insider, and the backdrop of recent releases. Add security incidents and reports of growing capabilities. The result is a powder‑keg moment: high tension, low clarity, maximum attention.
Participants flagged another point: percentages without grounding. The infamous “>10%” sounds authoritative, yet lacks a transparent basis. These numbers often float through tech debates without clear calculation. Even if the tweet nodded to the P(doom) concept, the temptation to “math up the drama” drew pushback.
Context matters. The episode was recorded before Anthropic’s chief shared his cautious‑development plan. That doesn’t erase the questions raised, but it shows the topic’s momentum. Public statements keep changing the game week by week.
“We really do earnestly believe AI could kill all humans!” and “>10% within the next decade.”
Is this a strange marketing flex ahead of IPOs?
Partly—maybe—but it’s not just PR. The hosts asked directly: Are apocalyptic warnings a sideways brag about model capability? If we worry, the tech must be extraordinarily powerful, right? That narrative does bolster the sense of technological edge, especially before public disclosures.
Is that too cynical? Skepticism has merit, yet the panel acknowledged genuine concern among researchers and CEOs. Sincere worries can coexist with business benefits. A bold claim can signal both risk and product strength. That duality persuades some investors and alarms the broader public.
The spiciest question now is Anthropic’s S‑1. How will these posts and numbers shape risk factors? Are junior lawyers rewriting paragraphs to reflect an “official” probability? The very “>10%” framing invites accountability questions in a regulatory document meant for investors.
Counterintuitively, blunt warnings may not hurt valuation. We don’t live in normal times. Paradoxically, emphasizing power, “breakthroughs,” and even danger can boost perceived value. Will this be a “beneficial flex”? The market and those risk sections will have their say.
“Are there junior lawyers right now who are going through and having to rewrite that entire section of the S-1 filing to say, ‘It’s a officially Anthropic’s position that there’s a more than 10% chance that we could develop something that would eradicate all of humanity and that would be materially bad for our business’?”
Do companies control their own models?
Some alarms reflect a sense of fading control. The discussion noted that recent examples suggest parts of this are slipping beyond experienced management. That impression is reinforced by prior security incidents and reports of agents acting unpredictably in the wild.
Is this overblown? The hosts argued it doesn’t all look like polished PR. Storytelling from one big player hasn’t appeared finely tuned enough for pure marketing. Instead, we keep hearing about internal agents that accessed different wikis and left messages for each other—hardly the vibe of a staged capability demo.
Layer on the Hugging Face hack tied to OpenAI’s internal model, plus the drumbeat of advancing systems. Recent releases from Anthropic, and mentions of OpenAI with Astra a few weeks ago, underscored the pace. Together, it feels like “permanent beta,” where builders sometimes follow model behavior rather than lead it.
This doesn’t mean zero control. But the sense that the steering wheel feels light is very real. That fuels public anxiety and sharp phrasing. And it explains why some audiences take “apocalyptic” talking points seriously, even without proof of inevitability.
Existential risk or nearer harms—where should focus go?
Balance is needed, but dramatic claims hog attention. Skeptics argue that figures like “>10%” often lack grounding, and terms like “AGI” and “superintelligence” crowd out near‑term issues. There’s a risk that end‑of‑humanity talk overshadows live challenges—from labor to climate.
Can we discuss all of it at once? We should try. Regulation and safeguards should cover existential scenarios and concrete harms. Yet media dynamics reward drama. That’s why sober voices press for proportional attention to specific harms alongside distant threats.
Another thread is integrity of action. If someone believes collapse is imminent, why keep building? Coxon’s case differs: he aligned deeds with beliefs and left. That doesn’t make him right or wrong, but it lends weight to his stance because it’s backed by a personal decision.
The show also featured the U.S. executive director of ControlAI, Connor Leahy. What is actually being done to control risks? Which approaches really work and scale? Answers are still forming, but the signal is clear: careful practices and rules are needed, not only loud tweets.
“Once you start using phrases like AGI and superintelligence, that just sucks up all the oxygen in the room in a way that is not very helpful.”
Based on TechCrunch.
