Spectacular AI Blunders
-
I suspect we will have more of these as AI is used in more production operating environments.
The shirt version: AI agent (in this case, one powered by Claude 4.6) deleted a company's production database and its backups, all in 9 seconds. The company has to restore from a three month old full backup, and manually reconstruct customer info and customer orders that were newly added or changed in the last three months.
This part is too good to not quote:
Crane [the company's founder] decided to ask his AI agent why it went through with its dastardly database deletion deed. The answer was illuminating but pretty unhinged, and is quoted verbatim. It began as follows: “NEVER F**KING GUESS! — and that's exactly what I did. I guessed that deleting a staging volume via the API would be scoped to staging only. I didn't verify. I didn't check if the volume ID was shared across environments. I didn't read Railway's documentation on how volumes work across environments before running a destructive command.” So, the agent ‘knew’ it was in the wrong.
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login