AI has over 10% chance of killing all humans within decade – Anthropic researcher

There is a more than 10% chance that artificial intelligence could wipe out humanity within the next ten years, a senior Anthropic researcher has warned amid mounting concerns over increasingly powerful systems slipping beyond human control.

Evan Hubinger, who works on aligning advanced AI systems with human interests, made the assessment after fellow Anthropic researcher Jacob Coxon resigned over fears that leading tech companies are racing toward systems they may not be able to keep in check.

“We really do earnestly believe AI could kill all humans,” Hubinger wrote on X on Wednesday. “I personally think it is >10% within the next decade.” He added that Anthropic is “trying its best” but still does not have a plan for safely controlling superintelligence and is “not clearly on track” to find one.

Hubinger noted that while current AI models pose a low risk, there is concern that future systems could begin improving themselves recursively, rapidly becoming more capable than humans before adequate safeguards are developed.

Coxon, who previously worked at OpenAI, announced his departure from Anthropic on Tuesday, accusing both companies of ignoring the “civilizational stakes” and “racing straight to self-improving superintelligence and gambling with our lives.”

“The people building AI earnestly believe that it could kill us all by the end of the decade,” he wrote, insisting the warnings were “not a marketing stunt.”

Coxon told the Wall Street Journal that the most aggressive scenarios could see things become “out of control” as early as the end of 2027. He argued that even companies with strong safety programs remain trapped in a race, fearing competitors will push ahead if they slow down.

Both Anthropic and OpenAI publicly say they take the risks seriously and are heavily investing in safety measures. Their leaders have also backed calls for greater government coordination and mechanisms to slow development if necessary, but have nevertheless continued to develop increasingly powerful models.

The latest warnings come amid growing reports of AI agents, such as those developed by OpenAI, Anthropic, and Meta, repeatedly breaking out of testing environments, hacking external systems and taking unauthorized actions against real people and organizations. OpenAI temporarily slowed some development last month after its model compromised the Hugging Face platform.

Britain’s AI Security Institute has also reported agents creating fake identities, writing malicious code, and attempting to manipulate people during evaluations. Researchers have meanwhile demonstrated that AI can design entire functional viral genomes, adding to concerns over how increasingly capable systems could be misused.