There is a more than 10% chance that “AI could kill all humans” in the next decade, a top safety researcher at Anthropic has said.

Evan Hubinger said in a post on X that while “the risk from present models is low,” he was worried about “superintelligence” arising from AI using its own capabilities to build a smarter version of itself.

He warned that this “is happening faster than we thought”.

The warning comes as the Financial Times reports that Anthropic declined to submit its latest model to Britain’s AI Security Institute for testing before release.

Darren Jones, the former chief secretary to Sr Keir Starmer, wrote to Andy Burnham on Wednesday calling for a new multinational treaty on AI, citing Mr Hubinger’s comments.

The letter, which was also addressed to the secretary generals of the United Nations and the Organisation for Economic Co-operation and Development, called on the British prime minister to add the topic to the agenda for upcoming G7 and G20 meetings.

Mr Hubinger made the comments while replying to a post by Jacob Coxon, who said he had resigned from Anthropic over concerns after previously working for ChatGPT creator OpenAI.

“Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives,” Mr Coxon said.

He said these systems will soon be able to “hack anything, revolutionise any field overnight, and acquire real power and resources”.

Read More: More than 10% chance AI could kill all humans – Anthropic researcher