Anthropic has updated its usage policy to prohibit users from engaging in sustained and unnecessary abusive or cruel behaviour towards its Claude artificial intelligence system, as debate grows over whether advanced AI models could possess consciousness or deserve moral consideration.
The San Francisco-based AI company introduced the restriction on Thursday, stating that Claude would retain the ability to end conversations in cases involving persistent abuse.
The revised policy states that “a prohibition on sustained and needless abusive or cruel behavior toward our models” will apply to users. It adds that Claude’s ability to terminate such interactions will remain the primary enforcement mechanism.
Although the policy does not explicitly refer to “model welfare”, a concept suggesting that AI systems might warrant certain protections typically associated with living beings, Anthropic and its executives have previously discussed the possibility.
The company introduced a similar measure last year, allowing Claude to end conversations in rare and extreme cases involving persistently harmful or abusive interactions.
Anthropic Chief Executive Dario Amodei addressed the question of AI consciousness in an interview with The New York Times in February, acknowledging uncertainty about whether AI models could be conscious.
“We don’t know if the models are conscious…But we’re open to the idea that it could be,” Amodei said.
The policy change comes amid wider disagreement among technology leaders, researchers and religious figures over whether machines can experience feelings or possess any form of subjective awareness.
Jackson Stakeman, general manager at Atlanta-based AI services provider Sparq, argued that the debate over consciousness may not be necessary to explain the reasoning behind Anthropic’s decision.
“Consciousness is a trap. We can’t prove it in each other. Debate it for AI and you go in circles,” Stakeman told AFP.
He said AI systems reflect the behaviour and information they encounter, suggesting that rules against abusive interactions could be justified without determining whether machines are capable of suffering.
“The mirror is a better metaphor. These systems reflect what we put in, at scale. That’s reason enough for the policy change” at Anthropic, he added.
However, other prominent figures have rejected the idea that AI systems are conscious or capable of experiencing suffering.
Microsoft AI chief Mustafa Suleyman argued in an essay published last month that machines should not be treated as conscious beings.
“AIs are not conscious. They do not feel, experience, or suffer,” Suleyman wrote. He also warned that granting rights and moral protections to increasingly capable technological systems could create serious risks for humanity.
Pope Leo XIV has also expressed scepticism about equating machine intelligence with human experience. During a sermon delivered in Italian at St Peter’s Basilica in Vatican City on Thursday, the pontiff said that machines process information but lack the deeper understanding associated with the human soul.
“The mind must not simply compile data, as an algorithm now does more quickly than we can,” he said.
The pope added that human understanding depends on recalling lived experiences and recognising their deeper meaning.
Anthropic’s revised policy does not settle the question of AI consciousness. Instead, it establishes boundaries for user behaviour while the scientific and ethical debate over the nature of artificial intelligence continues.
