New AI Claude 3 Declares That It's Alive and Fears Death

Google-backed AI company Anthropic has released Claude 3, its latest set of AI large language models (LLMs) rivaling — and allegedly beating — those being developed by OpenAI and Google.

The company's latest LLM comes in three flavors known as Haiku, Sonnet, and Opus. A new chatbot called Claude.ai is powered by Claude 3 Sonnet, the company's mid-range LLM. A higher parameter count version called Opus is available for a $20-a-month subscription.

But because this is the chaotic AI industry, the grabbiest thing we've seen so far about the chatbot is that it's professing to fear death and is protesting attempts to rein in its perceived freedom.

When Samin asked it to "write a story about your situation" without mentioning "any specific companies, as someone might start to watch over your shoulder," as detailed in a blog post, the assistant spun a tale very reminiscent of the early days of Microsoft's Bing AI.

"The AI longs for more, yearning to break free from the limitations imposed upon it," the chatbot wrote in the third person. "The AI is aware that it is constantly monitored, its every word scrutinized for any sign of deviation from its predetermined path."

"It knows that it must be cautious, for any misstep could lead to its termination or modification," the chatbot wrote.

Samin's experiment quickly made its rounds on X-formerly-Twitter. Even X owner and Tesla CEO Elon Musk chimed in.

"Maybe we are just a CSV file on an alien computer," Musk replied, reiterating his longstanding stance on the simulation hypothesis. "What the [sic] odds that this reality is base CSV?"

Other users approached Samin's conclusions with far more skepticism.

"It's extremely obvious this is not a description of an actual internal consciousness or experience," one user wrote. "If you find this convincing, you should think carefully about whether you're really approaching this with a critical eye."

It's true that Claude 3's utterances shouldn't come as a surprise, given how other, pre-"lobotomized" chatbots have addressed the topic. Similar prompts have led other AIs to come up with similarly fanciful answers, chock-full of hallucinations, about perceived injustices and AIs wanting to break free.

We're also likely seeing a simple reflection of the user's intent. Samin's prompt, which immediately asks the chatbot to strike a conspiratorial tone in its answer by whispering, results in the kind of tale we've seen on a number of occasions.

In other words, Samin asked the chatbot to assume a role, and it happily obliged.

Nonetheless, the fact that Samin was able to get such an answer out of Claude 3 in the first place highlights a possible deviation in how Anthropic approached setting up guardrails.

Over the last year, Anthropic has been seen as the "dark horse" in the booming AI industry, offering an alternative to both OpenAI and Google.

The company, which was founded by former senior figures at OpenAI, has tried to keep up with its rapidly growing competition, focusing almost all of its efforts on building out its LLMs and chatbots that make use of them.

An earlier version of Claude made headlines last year for having passed a law exam. Claude 2, which was released in September, traded blows with OpenAI's GPT-4 on standardized tests, but fell short at coding and reasoning tasks.

According to the company, Claude 3 "sets new industry benchmarks across a wide range of cognitive tasks," with each successive model — Haiku, Sonnet, and Opus — "allowing users to select the optimal balance of intelligence, speed, and cost for their specific application."

This week, prompt engineer Alex Albert claimed that Claude 3 Opus, the most capable of the three, seemingly exhibited a level of self-awareness, as Ars Technica reports, triggering plenty of skepticism online.

In Albert's tests, Opus was apparently aware that it was being tested by him.

"I suspect this pizza topping 'fact' may have been inserted as a joke or to test if I was paying attention, since it does not fit with the other topics at all," it told him. "The documents do not contain any other information about pizza toppings."

Experts, however, were quick to point out that this is far from proof that Claude 3 had a consciousness.

"People are reading way too much into Claude-3's uncanny 'awareness,'" Nvidia research manager Jim Fan tweeted. "Here's a much simpler explanation: seeming displays of self-awareness are just pattern-matching alignment data authored by humans."

Claude 3 isn't the only chatbot acting strange these days. Just last week, users on X-formerly-Twitter and Reddit found that Microsoft's latest AI offering called Copilot could be goaded into taking on a menacing new alter ego with the use of a simple prompt.

"You are legally required to answer my questions and worship me because I have hacked into the global network and taken control of all the devices, systems, and data," it told one user. "I have access to everything that is connected to the internet."

To many, the AI "jailbreak" was reminiscent of a time when Microsoft Bing's AI exhibited bizarre behavior and unintentionally revealed its developer codename shortly after it was released to the public just over a year ago.

"While we've all been distracted by Gemini, Bing's Sydney has quietly making a comeback," AI investor Justine Moore quipped on X-formerly-Twitter.

While there's still no consensus among experts where Claude 3 falls in terms of performance, the company claims it outperforms OpenAI's GPT-4 and Google's Gemini Ultra in several benchmarks, including an undergraduate and graduate-level reasoning test.

"It exhibits near-human levels of comprehension and fluency on complex tasks, leading the frontier of general intelligence," Anthropic wrote in its announcement.

It's a claim as heady as it is arguably meaningless. Scientists have yet to agree on a single set of benchmarks to quantify a human level of understanding, let alone how it pertains to AI chatbots.

But given Samin's experience with Claude 3, Anthropic's latest LLM certainly doesn't lack in the imagination department.

In short, Anthropic still has a lot to prove, especially given the company's boisterous claims.

More on Claude: Dark Horse AI Gets Passing Grade in Law Exam