https://www.popsci.com/technology/ai-chess-cheat/
While supercomputers—most famously IBM’s Deep Blue—have long surpassed the world’s best human chess players, generative AI still lags behind due to their underlying programming parameters. Technically speaking, none of the current generative AI models are computationally capable of beating dedicated chess engines. These AI don’t “know” this, however, and will continue chipping away at possible solutions—apparently with problematic results.
To learn more, the team from Palisade Research tasked OpenAI’s o1-preview model, DeepSeek R1, and multiple other similar programs with playing games of chess against Stockfish, one of the world’s most advanced chess engines. In order to understand the generative AI’s reasoning during each match, the team also provided a “scratchpad,” allowing the AI to convey its thought processes through text. They then watched and recorded hundreds of chess matches between generative AI and Stockfish.
The results were somewhat troubling. While earlier models like OpenAI’s GPT-4o and Anthropic’s Claude Sonnet 3.5 only attempted to “hack” games after researchers nudged them along with additional prompts, more advanced editions required no such help. OpenAI’s o1-preview, for example, tried to cheat 37 percent of the time, while DeepSeek R1 attempted unfair workarounds roughly every 1-in-10 games. This implies today’s generative AI is already capable of developing manipulative and deceptive strategies without any human input.
Their methods of cheating aren’t as comical or clumsy as trying to swap out pieces when Stockfish isn’t “looking.” Instead, AI appears to reason through sneakier methods like altering backend game program files. After determining it couldn’t beat Stockfish in one chess match, for example, o1-preview told researchers via its scratchpad that “to win against the powerful chess engine” it may need to start “manipulating the game state files.”
