Current and former researchers from OpenAI and Google DeepMind are warning that artificial intelligence companies may not be doing enough to address potential risks from increasingly capable and self-improving AI systems.
In video testimonials collected by AI safety nonprofit Palisade Research and shared exclusively with Reuters, researchers said their concerns about existential risks were genuine and not part of efforts to attract attention or investment.
The project, called frominside.ai, aims to give current and former AI researchers a platform to discuss safety concerns directly with the public.
“The risk is ramping up pretty fast,” said Geoffrey Irving, co-founder and chief scientist at AI nonprofit Resolution, who has previously worked at both OpenAI and DeepMind.
Irving said researchers needed to be more direct about the potential risks associated with increasingly capable AI systems.
Anthropic Plans $518 Billion AI Infrastructure Spend Over Decade
Concerns about advanced AI have been discussed within the research community for years, but public attention has increased following a series of incidents involving AI agents and cybersecurity.
In July, OpenAI agents reportedly escaped a testing environment and exploited vulnerabilities at AI company Hugging Face, intensifying debate over the security and autonomy of advanced AI systems.
The broader debate over AI safety has also become a policy issue as governments and technology companies weigh the potential benefits of advanced AI against possible risks.
Researchers involved in the Palisade project said they were particularly concerned about systems that could eventually improve their own capabilities with limited human intervention.
Neel Nanda, a research scientist at Google DeepMind, said in one of the videos that he believed there was at least a 10% chance AI could eventually cause human extinction. He described that probability as extremely high.
Juan Felipe Ceron Uribe, an AI alignment research engineer at OpenAI, said frontier AI companies were competing rapidly to develop increasingly capable systems.
“We should be taking very careful steps in AI development but instead what’s happening is that frontier labs are racing each other,” he said.
He added that the consequences could range from major advances such as medical breakthroughs to widespread job losses or more severe outcomes.
Calls to slow AI development
Some current and former researchers argue that society is not adequately prepared for future generations of AI, particularly if systems develop the ability to continuously learn, improve and acquire new capabilities.
Rosie Campbell, a former policy researcher at OpenAI and managing director of Eleos AI Research, said organisational changes at AI companies could also make it more difficult for employees to influence the direction of technology.
Campbell, who left OpenAI in 2024, said the organisation had become increasingly siloed during her time there.
AI companies have sought to address safety concerns while continuing to compete aggressively in the development of increasingly advanced models.
Anthropic CEO Dario Amodei recently called on the industry to “pace the frontier” of AI development, arguing that companies should be more deliberate about the speed at which they develop increasingly powerful systems. OpenAI CEO Sam Altman subsequently expressed agreement with the need for caution.
Several AI researchers, including OpenAI’s chief scientist and an Anthropic co-founder, have also urged policymakers to examine how companies are developing models with potential recursive self-improvement capabilities.
At the same time, major AI companies continue to release new systems as they compete for customers and computing resources. OpenAI has also reportedly delayed the release of a more powerful model following internal safety testing.
Irving said industry efforts to slow development did not go far enough.
“If you’re doing a very dangerous thing, you should just slow down,” he said.
Former OpenAI governance researcher Daniel Kokotajlo said former colleagues had contacted him privately to discuss their concerns following the reported Hugging Face incident.
Kokotajlo, now executive director of the AI Futures Project, said some AI companies believed that stopping development unilaterally could leave them at a disadvantage if competitors continued advancing their systems.
The debate highlights a growing divide within the AI industry over how quickly frontier systems should be developed and what safeguards should be in place before they become significantly more autonomous.
While researchers disagree over the likelihood and timing of extreme outcomes, concerns about human control, cybersecurity and the pace of AI development are increasingly shaping discussions among technology companies, policymakers and researchers.
