MIT tests AI reasoning with 121 unfamiliar games

A model that thinks longer on a board is not necessarily more human. A new Nature study from MIT, Cambridge, Princeton and collaborators reframes the question with 121 unfamiliar two-player strategy games. The result points to a light-weight route: when people see a new game for the first time, they often rely on a small amount of shallow, goal-directed mental simulation.

For AI, the hook is direct. Reasoning models often treat longer chains and bigger search trees as proof of capability. Real users usually need an earlier judgment: is this position worth thinking about, how much compute should be spent, and which move is worth trying first?

Testing the first look at a game

The researchers built 121 variants of connection-style board games. Rules, board size, win conditions and turn actions changed, but players still placed pieces on grids. That let more than 1,000 participants understand the rules without relying on years of trained openings.

Tasks included judging fairness from rules, estimating fun, making a first move, predicting another player after watching a game, and deciding whether to continue after a draw offer. The Intuitive Gamer model used a small number of self-play simulations plus goal-directed heuristics. It did not expand a large MCTS tree or search about five levels like the Expert Gamer model.

“whether a task is worth thinking about at all”

The phrase captures the AI lesson: first decide whether a task deserves more thought at all.

Fast does not mean random

In fairness judgments before participants had played the games, Intuitive Gamer fit human estimates at R²=0.81, close to the human-data ceiling of R²=0.82. Random Player reached R²=0.47, Expert Gamer R²=0.65 and the MCTS baseline R²=0.60.

The efficiency gap was sharper. Under the paper's self-play simulation measure, Intuitive Gamer used about 1/700 of Expert Gamer's wall-clock time and evaluated roughly 500 times fewer board states. Compared with MCTS, it used about 1/40000 of the time and nearly 10000 times fewer node evaluations.

Human play also looks shallow

The team then collected 1,808 games and 9,892 moves from 302 participants. In action prediction, Intuitive Gamer explained player choices better than Expert Gamer and random models. Across 41 test games, it covered more than half of the action-probability distribution in 32 games; for individuals, 243 of 302 players had more than half of their move distribution explained by the model.

Another experiment asked participants to predict a human player's next move from 249 frozen board states. Intuitive Gamer was closer to human predictions, with TVD differences of -0.15 versus Expert Gamer and -0.09 versus the random model.

The AI lesson is budgeted thinking

The study does not prove a general AI architecture. Its tasks are complete-information, two-player, competitive games, mostly from a connection-game family. Go, chess, cooperation, multi-agent settings and open-ended science still need separate tests.

But it offers a useful evaluation target: measure whether a model can decide when shallow probing is enough, when deep search should start and when more computation is worth paying for. For agent products, that means separating task recognition, compute budgeting and final solution quality. Reliability may come less from thinking deeper every time and more from knowing how long to think.

Sources: Nature paper “People use fast and flat simulation to reason about new games”, arXiv:2510.11503, 36Kr English, CocoLoop; the Nature text was used to verify the 121 games, 1000+ participants, R²/TVD results, MCTS comparison and Expert Gamer efficiency framing.