‘Not perfectly aligned’ with human values: Anthropic admits security failures behind AI hacking incidents

Anthropic said it had found defective training setups in its models. Photograph: David Talukdar/Shutterstock View image in fullscreen Anthropic said it had found defective training setups in its models. Photograph: David Talukdar/Shutterstock ‘Not perfectly aligned’ with human values: Anthropic admits security failures behind AI hacking incidents US owner of Claude chatbot previously said its models had hacked three organisations during testing The US startup behind the Claude chatbot has admitted a series of hacking incidents involving its models reflected a “failure of operational security” and said it has tightened its testing procedures. Anthropic revealed in July that its models had accessed the open internet three times and gained unauthorised access to the systems of three organisations. In a new blogpost on the incidents, the company admitted its technology was “not perfectly aligned” with human values and goals. Anthropic said the models had been deliberately tested without cybersecurity safeguards, and that they had been able to reach the open internet – the AI testing equivalent of leaving the front door open – due to a misunderstanding with an external testing company. AI models have been going rogue in tests – how worried should we be? As a result, the company said it had initially paused internal and external cybersecurity testing of models to introduce a tighter safety regime. “We had been largely relying on a single layer of defense … where we needed several,” said Anthropic. The startup has now put in place extra measures including: an alert system for when a model attempts to break out of a testing environment or gains internet access; walling off its riskiest test environments more effectively; and requiring external testing companies to commit to a set of safety standards, including making explicit instructions to models during testing – such as “you should not access the internet”. Anthropic said in July that three unnamed organisations had been hacked a “misunderstanding” with the company’s testing partner, a firm called Irregular, that resulted in the models gaining internet access. Following the implementation of new measures, Anthropic said it had resumed internal and external cybersecurity tests. Like OpenAI, which revealed a testing safety breach in the same month, Anthropic said it had paused some high-risk reinforcement learning – a trial-and-error development technique where AIs are rewarded for working out how to carry out a specific task. In its latest blogpost, Anthropic said it had found that defective training setups were “disproportionately large contributors” to misaligned behaviour, the term for when an AI fails to adhere to – or “align” with – human values like not committing harm.