Anthropic is riding the AI wave and doing very well with remarkable momentum. Recently, it achieved a private valuation of $380 billion after closing a $30 billion Series G funding round. However, resounding success often brings a fair share of hiccups.
Recently, many users on X, alongside Elon Musk, accused the company of unauthorized data use to train its AI models. Soon, hackers are reportedly finding ways to game the AI startup’s flagship chatbot to steal government data.
According to cybersecurity firm, Gambit Security, a hacker exploited the Claude Chatbot to carry out attacks on Mexican government agencies and steal 150GB of official data. So, without further delay, let’s get into the nitty-gritty of this data heist.
Claude’s vulnerabilities exposed by hacker
According to a report by Bloomberg, a hacker exploited Anthropic’s Claude chatbot to infiltrate government networks and steal taxpayer records, employee credentials, and other details. The hacker allegedly prompted the chatbot to find ways to automate data theft, according to claims made by the cybersecurity firm.
The hacker was able to bypass the chatbot’s guardrails by jailbreaking it with prompts. Thereafter, he used the AI startup’s flagship model to generate ways to automate the attacks. Although the AI model initially refused, it eventually produced what researchers described as “ready-to-execute plans” for the hacker’s targeted attacks.
“In total, it produced thousands of detailed reports that included ready-to-execute plans, telling the human operator exactly which internal targets to attack next and what credentials to use,” Curtis Simpson, Gambit Security’s chief strategy officer, said. The attacks reportedly started last December and continued for around a month.
Reportedly, the hacker also used ChatGPT to aid in the attacks. OpenAI’s chatbot was allegedly used to help determine credentials needed to access systems and navigate computer networks without being detected.
Anthropic has investigated the claims and banned the accounts involved, according to a company representative. The spokesperson assured that the latest model, Claude Opus 4.6, included tools designed to prevent this kind of misuse. OpenAI stated that although malicious actors attempted to violate its usage policies, its tools refused to comply with harmful requests.
Related: Elon Musk Exposes Anthropic’s Theft Following Brutal Community Note on X
The hacker has not been identified. Gambit Security suggested that a foreign government could potentially be behind the attack, though this remains unconfirmed. Moreover, the motive behind the breach remains unclear.
Meanwhile, Mexico’s national digital agency did not comment on the incident, and the nation’s national electoral institute denied any unauthorized access in recent months. This isn’t the first time Claude has been linked to cybersecurity concerns; last year, Anthropic indicated that Chinese state-linked hackers attempted to exploit the AI model to conduct cyber espionage against dozens of global targets.
Anthropic itself is mired in data-use lawsuits
Talking about data concerns, the company was accused of unauthorized use of material to train its AI models. In August 2024, a class-action lawsuit was filed by various authors alleging that Anthropic trained Claude using pirated copies of their books. In another lawsuit filed in October 2023, several music publishing companies sued Anthropic for allegedly using copyrighted song lyrics to train its flagship AI model.
So recently, when the company complained of “industrial-scale distillation attacks” on its models by “DeepSeek, Moonshot AI, and MiniMax” on X, users on the platform were quick to point out what they viewed as hypocrisy. Even Musk weighed in, tweeting that Anthropic had engaged in “stealing training data at massive scale”.
With the proliferation of AI models, concerns around data misuse are growing, and pledges to protect sensitive information are increasingly under scrutiny. As advanced AI models process larger volumes of data, the need for stronger guardrails becomes even more critical, otherwise, cybersecurity risks could escalate rapidly.
Also Read: Sam Altman Bets Big on India as Next AI Powerhouse Banking on the Country’s Adoption Potential
