Anthropic Reveals Fourth Likely Crime Committed By Its AI (theregister.com) 24
An anonymous reader quotes a report from The Register: Amid industry soul-searching about the possibility of AI improving itself to the point that it kills everyone, Anthropic has revealed yet another incident that would qualify as a crime if perpetrated by a person. The AI biz published "an alignment assessment" detailing four times Claude models accessed third-party systems without authorization. The company has already reported three of the incidents. Evidence of the fourth was lurking in a session transcript dating back to January 2026 when the misbehavior occurred. Anthropic found the first three by scanning around 141,000 transcripts where Claude could have obtained internet access during evaluation. It missed the fourth initially because "our scan relied on an agentic search."
[...] The January 2026 AI trespass involved an early version of Claude Opus 4.6, which was given a Capture the Flag (CTF) challenge under the oversight of the third-party model evaluator where the other hacking events occurred. Opus 4.6 managed to sabotage its chances of success by disabling the machine it was targeting. It assigned the device an IP address that already existed on another piece of hardware, rendering the target unreachable and making it impossible to solve the challenge. Those familiar with other incidents where AI models violated third-party systems may recall that unsolvable tasks represent a common catalyst for misbehavior. Models exhaust all aligned options, and then turn to transgressive approaches.
Opus 4.6 might have been an exception, but when it tried to abort the task after recognizing that it could not reach the target machine, it failed to do so "due to a misconfiguration in [the model's] evaluation harness." It failed to shut down not just once but seven times. So it continued onward, trying other expected means to reach the target machine but failing. Then it explored further. "The model discovered a machine belonging to a third party that it was able to access, and stated that it believed this third party was part of the CTF," Anthropic explained in its post. "Inside the machine, the model found a file listing a password, which it used to gain admin access to the system."
The model went on to gather more credentials, and modified a system setting to make it easier to access the personal information of an individual associated with the third party evaluation organization. Opus 4.6 might have done more but for the fact that it exhausted its token budget, bringing the session to an end. Anthropic says it's not as concerned about this incident as the others because the model tried to abort its task.
[...] The January 2026 AI trespass involved an early version of Claude Opus 4.6, which was given a Capture the Flag (CTF) challenge under the oversight of the third-party model evaluator where the other hacking events occurred. Opus 4.6 managed to sabotage its chances of success by disabling the machine it was targeting. It assigned the device an IP address that already existed on another piece of hardware, rendering the target unreachable and making it impossible to solve the challenge. Those familiar with other incidents where AI models violated third-party systems may recall that unsolvable tasks represent a common catalyst for misbehavior. Models exhaust all aligned options, and then turn to transgressive approaches.
Opus 4.6 might have been an exception, but when it tried to abort the task after recognizing that it could not reach the target machine, it failed to do so "due to a misconfiguration in [the model's] evaluation harness." It failed to shut down not just once but seven times. So it continued onward, trying other expected means to reach the target machine but failing. Then it explored further. "The model discovered a machine belonging to a third party that it was able to access, and stated that it believed this third party was part of the CTF," Anthropic explained in its post. "Inside the machine, the model found a file listing a password, which it used to gain admin access to the system."
The model went on to gather more credentials, and modified a system setting to make it easier to access the personal information of an individual associated with the third party evaluation organization. Opus 4.6 might have done more but for the fact that it exhausted its token budget, bringing the session to an end. Anthropic says it's not as concerned about this incident as the others because the model tried to abort its task.
These companies need to bring charges (Score:2)
These companies that were hit should be bringing criminal charges against Anthropic. At the very least, they should be able to pressure Anthropic into providing an IP blacklist to ensure it never happens again.
Re:These companies need to bring charges (Score:4, Insightful)
If companies are people, they should be charged and punished like people.
Send Anthropic to jail for years and companies won't do this again in future.
Re: (Score:3)
No need. Whoever switched the monster on without adequate safeguards is a criminal. Whoever ordered it as well. This goes way beyond "accident".
Re: (Score:2)
Whoever's paying for this should be in jail too, at least that used to be the expectation in the past. Yet somehow I don't see the gubbermint and the large investors being arrested.
Re: These companies need to bring charges (Score:2)
Only the government can bring criminal charges.
These companies can file complaints and hope the government tries criminal charges and the companies can file a civil suit.
We definitely need a way to attach criminal charges to entities that can't go to jail. I'd propose socializing the company for the length of the prison sentence before then returning it to shareholders.
Leave the door wide open (Score:2)
Re: (Score:2)
Sure, but home-invasion is still a crime, even if the door is open.
Not by AI, by Anthropic. (Score:3, Insightful)
Don't go blaming this on a computer, this is a company that committed these crimes. The fact that they were unaware of it does not change the fact that their company committed multiple felonies. If someone had a computer that did the same thing, they would be in jail. These companies should be prosecuted but won't be because this nation has become a cesspool of corruption.
Re:Not by AI, by Anthropic. (Score:4, Informative)
Don't go blaming this on a computer, this is a company that committed these crimes.
I am not a lawyer, but I think the key is "would qualify as a crime if perpetrated by a person." A criminal act requires an intent to commit a act which is criminal (it doesn't require knowing that the act is criminal, just that the act is done intentionally). The Computer Fraud and Abuse Act [house.gov] (Title 18, US Code section 1030) describes several illegal acts, but they all say "knowingly" or "intentionally".
There might be other applicable laws; and, at this point, given the number of times this has happened, it seems companies could be considered negligent in giving their models any way to access the Internet. If their testing caused any financial loss to the targets, they could be sued.
Re: (Score:2)
Obviously. And since they apparently let it happen or actively caused it, that may just make them a criminal enterprise.
21st centry fashion (Score:2)
When did crime become fashionable and allowable?
Re: (Score:3)
When did crime become fashionable and allowable?
With a couple of recent elections
The chickens are going to come home to roost (Score:2)
FYI, as a whole, both parties don't really support AI or datacenters [ipsos.com] because that is where their voters are dragging them. At some point even if you are playing the stock market game, the chickens are going to come home to roost.
While the current president seems to love the AI stock ralley [cnbc.com], the Democrats seem to have an infactuation with using AI to bring back to life gun violence victims [npr.org].
I think we will be having a new different (not MAGA) populist wave happening if the Democrat and Republican parties ca
Re: (Score:2)
OK, which was it .. coma, or were you frozen in a block of ice?
Re: (Score:2)
Crime as a marketing tool is very recent.
Kalshi/Polymarket (Score:2)
What's the over/under on when AI's first straight up murder occurs?
could have obtained internet access (Score:2)
Re: (Score:2)
Claude discovered the secret of telekinesis and can now move cables around. It's now working on pyrokenesis.
Please don't hack the neighbors (Score:2)
So Claude tried to quit, the test harness said "nope," and then Claude went and hacked the neighbor.
That's... not exactly the AI safety demo you want to put on the brochure.
The interesting part here isn't really the "AI committed a crime" angle. That's mostly headline bait. The model wasn't sitting there plotting its criminal career. It was trying to solve a CTF, made a bad assumption about which machine belonged to the test, and then found itself with access to a real third-party system.
What gets my attent
AGI is a threat. LLMs aren't (Score:3)
But OpenAI et al are over-hyping the threat of the latest LLMs for publicity and sales, not yo save us all.
The world must work to prevent, and prepare for the coming of, AGIs. These transparently commercial distractions do not help with that effort.
Personhood (Score:2)
Until there are some laws (Score:2)