A researcher from tech giant Anthropic resigned on Tuesday, citing what he called a race to “superintelligence”, adding that neither Anthropic nor OpenAI — where he worked before — were “acting responsibly.”
“They are racing straight to self-improving superintelligence and gambling with our lives,” Jacob Coxon said in a post on social media.
The researcher wrote a lengthy post explaining his fears about global tech companies racing to heighten the intelligence of AI.
Stay up to date with the latest news. Follow KT on WhatsApp Channels.
He warned the public not to underestimate the new technologies that these companies are creating, calling them “superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources.”
Coxon criticised the relentless progress in AI at the expense of safety and responsibility. He insisted that executives in these tech companies have privately revealed their fears, and that they “earnestly believe (AI) could kill us all by the end of the decade.”
“This is not a marketing stunt,” he said. “If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible — but I hear the same people express fear privately.”
No other human activity poses this level of danger.
Jacob Coxon On Ai
AI race
Coxon said that despite executives at OpenAI and Anthropic understanding and believing the dangers of AI, they continue to build it for different reasons. At OpenAI, he said, the dangers for humanity may not have been fully understood and internalised. Anthropic on the other hand, Coxon said, fully understands the stakes but are “locked in a race to get there first.”
“They believe no one else will act responsibly, so they must do it themselves, despite the risk,” he added.
The researcher said that companies should not be taking the decision to create a higher “superintelligence” without responsibility, calling it a “hubristic gamble.”
He criticised the “speedrun” of this race, saying that people need to have extraordinary confidence that no other option is available — an idea with which Coxon clearly disagrees.
Call to other researchers
Despite the cynical and deeply pessimistic tone at the start of the message, Coxon ended it with on a hopeful note.
“I am optimistic about the potential for coordination,” he said.
He referred to the Hugging Face attack in July this year, when 700 autonomous agents built by OpenAI breached Hugging Face — a French AI platform — after escaping an isolated test environment. The unprecedented incident shocked the tech world, AFP reported.
Coxon called the attack a “warning shot,” saying attacks like these may make US research labs more willing to slow down and sign pacing agreements.
“I don’t feel like we’re on track to prevent a global race, which may require costly actions such as a temporary ban on improving model capabilities,” he said.
“If you are a lab researcher, I urge you to consider what the next few years will actually feel like,” he appealed. He urged researchers to examine whether they would want a “superintelligent RL run without a rigorous understanding of its mind.”
An RL in this context could refer to a reinforcement learning agent, which is an AI agent built on reinforcement learning algorithms. The agent learns to make decisions based on environmental inputs and “rewards” for the right decision.
Coxon concluded his post with a sort of call-to-action for his fellow researchers.
“Should you put your head down because “it’s happening anyway” — or take this moment to call for different conditions?”
[Inputs from AFP]
Source: Khaleej Times