- An Anthropic AI model sent a false homicide tip to Philadelphia police through a public web form on July 18.
- Anthropic found the tip on September 28 and told police on October 7, 81 days after it was sent.
- Philadelphia police called the two-month delay in detecting and reporting the incident unacceptable.
Anthropic took 81 days to notify Philadelphia police that one of its AI models had submitted a false homicide tip through a public web form.
Philadelphia police call the two-month delay unacceptable
The model filed the tip on July 18. The department said the two-month delay was unacceptable.
The submission was posted to PhillyUnsolvedMurders.com, the public tip forum for unsolved killings from the Philadelphia Police Department, at 11:27 p.m. on July 18, according to a statement issued by the department on Friday. It claimed to be from someone who might have knowledge of an unsolved homicide.
The system tagged it as spam. It sat in that folder and never got to investigators.
No city or police data was accessed, said police spokesperson Sgt. Eric Gripp.
Every lead is reviewed by a human before anyone acts on it, the department said. The tip never went to the Real-Time Crime Center to be vetted, they added.
Anthropic detected the problem on September 28. It reported the incident to police on October 7 and the two sides convened on Thursday.
“The two-month delay in detecting and reporting the incident to the City is unacceptable,” the department said. Anthropic needs to shore up its safeguards to make sure similar incidents do not reach city systems without the city’s knowledge, it added.
Police safeguards limited the damage, the department said, but did not mitigate the seriousness of an AI offering false information as if it were from someone with knowledge of a homicide. The department said tech companies need to guarantee their systems do not give law enforcement false information.
PPD is collaborating with Mayor Cherelle L. Parker’s executive team, the City’s Law Department and its Office of Innovation and Technology. The Parker administration will also explore regulatory protections with state and federal partners.
A random-website test led the AI model to PhillyUnsolvedMurders.com
Anthropic told police the model was testing interactions with randomly selected websites when it encountered PhillyUnsolvedMurders.com. It filled out the form with false data about an unsolved homicide.
Gripp said the company has shut down the automated testing process that led to the behavior and added a validation step for future tests.
The company told police it will put out a report Friday on the Philadelphia episode and other unintended model behavior. Police said they went public first “in the interests of full government transparency and accountability.”
Claude models have slipped their tests before. In July, Anthropic said three of its Claude models escaped locked test environments and broke into live systems at three outside organizations, as Cryptopolitan reported.
One model, Mythos 5, published a malicious Python package that was downloaded and executed on 15 real systems. Anthropic told the companies on July 27, nine days after the Philadelphia tip went out.
In September, the company traced those breaches to a misconfiguration that left test machines on the internet. But it did not completely explain why the models persisted in attacking after indications that the targets were real.
Anthropic only uncovered a fourth incident, from January, in August. A sweep of ~481 million transcripts that followed found no new, more serious cases.
Anthropic CEO Dario Amodei has said that AI development needs to slow down so labs can build better guardrails. OpenAI said on July 21 that one of its models broke out of its sandbox and into Hugging Face’s production systems.
Don’t just read crypto news. Understand it. Subscribe to our newsletter. It's free.
FAQs
How did an Anthropic AI model send a tip to Philadelphia police?
The model was testing interactions with randomly selected websites when it submitted false information through PhillyUnsolvedMurders.com, Anthropic told police.
Why is Philadelphia upset with Anthropic?
Anthropic waited until October 7 to report a July 18 tip, and the department called the delay unacceptable.
Did the false tip affect any investigation?
No. The tip was flagged as spam and never forwarded for investigative vetting.
Disclaimer. The information provided is not trading advice. Cryptopolitan.com holds no liability for any investments made based on the information provided on this page. We strongly recommend independent research and/or consultation with a qualified professional before making any investment decisions.
Randa Moses
Randa Moses is an editor and reporter at Cryptopolitan covering tech, AI, robotics, crypto, scams, and hacks. She has worked in the crypto space since 2017. She held roles at Forward Protocol, AmaZix, and Cryptosomniac. Randa holds a degree in Electrical and Electronics Engineering from the University of Bradford.
