Artificial intelligence company Anthropic has updated its usage policy to prohibit sustained and needless abusive or cruel behaviour towards its Claude AI models, amid growing debate over whether artificial intelligence could be conscious.
The company’s revised policy states that Claude will retain the ability to end conversations in cases involving persistent abuse.
“Claude’s ability to end these interactions will remain the primary enforcement mechanism,” the policy says.
The update does not explicitly refer to “model welfare”, a concept that considers whether AI systems might deserve protections similar to those afforded to living beings. However, Anthropic and its executives have publicly discussed the possibility that advanced AI models could warrant moral consideration.
Anthropic did not immediately respond to a request for comment.
Anthropic Raises Questions About AI Welfare
The company introduced a similar safeguard last year, allowing Claude to end conversations in rare, extreme cases involving persistently harmful or abusive interactions.
LeBron James Says 76ers Need Time to Build Chemistry After Nets Defeat
In February, Anthropic chief executive Dario Amodei told The New York Times that he was uncertain whether AI models could be conscious.
“We don’t know if the models are conscious… But we’re open to the idea that it could be,” Amodei said.
The policy change comes as researchers and technology companies debate whether increasingly capable AI systems could develop experiences that resemble consciousness or suffering.
However, experts disagree sharply over whether current AI systems possess any form of subjective experience.
Experts Disagree Over Machine Consciousness
Jackson Stakeman, a general manager at Atlanta-based AI services provider Sparq, argued that consciousness may not be the most useful starting point for the debate.
“Consciousness is a trap. We can’t prove it in each other. Debate it for AI and you go in circles,” Stakeman told AFP.
He suggested that AI systems reflect human inputs and behaviour at scale, making safeguards against abusive interactions worthwhile regardless of whether machines can experience suffering.
“The mirror is a better metaphor. These systems reflect what we put in, at scale. That’s reason enough for the policy change at Anthropic,” he said.
Others have rejected the idea that AI systems could be conscious.
Microsoft AI chief Mustafa Suleyman wrote in an essay last month that AI systems do not feel, experience or suffer.
He also warned against granting rights or moral protections to technological entities that could eventually become far more capable and intelligent than humans.
Pope Leo XIV Questions AI’s Limits
Pope Leo XIV has also questioned whether machines can possess qualities associated with the human mind and soul.
During an Italian-language sermon at St Peter’s Basilica in Vatican City on Thursday, the pope said that human thought involves more than rapidly processing information.
“The mind must not simply compile data — as an algorithm now does more quickly than we can,” he said.
He added that the human mind draws on lived experiences and recognises deeper meaning through the human soul.
The contrasting views highlight an unresolved question in AI development: whether safeguards should focus solely on how technology affects people or also consider the possibility of future AI systems having morally significant experiences.
For now, Anthropic’s policy establishes a behavioural boundary for interactions with Claude without declaring that the system is conscious or capable of suffering.
