The University of California, Berkeley announced on October 9 a study on AI assistance and persistence. Participants who used ChatGPT to work through problems for about 10 minutes did worse once the tool was taken away: accuracy dropped, and the share of questions they skipped went up. The paper, "AI Assistance Reduces Persistence and Hurts Independent Performance," first appeared on arXiv in April and was updated to a fifth version on October 3. The peer-reviewed version was published at COLM, the Conference on Language Modeling.
The paper has five authors, including a researcher at Berkeley's Center for Human-Compatible AI, Brian Christian, author of The Alignment Problem, and collaborators from Carnegie Mellon, MIT, Oxford and UCLA.
How the three experiments worked
The researchers recruited 1,222 people online and ran three randomized controlled trials. The first two used fraction arithmetic; the third used SAT reading-comprehension practice questions. Each trial had two phases. In the practice phase, one group could use AI while the control group worked alone. In the following test phase, which had 3 questions, nobody had access to AI.
The results below are based on the samples remaining after exclusions:
| Experiment | Test accuracy (AI group / control) | Skip rate (AI group / control) |
|---|---|---|
| Fractions 1 (307 people) | 57% / 73% | 20% / 11% |
| Fractions 2 (585 people) | 71% / 77% | 10% / 7% |
| Reading comprehension (168 people) | 76% / 89% | 8% / 1% |
The accuracy gap was statistically significant in all three experiments. The skip-rate gap was significant in the first and third experiments but not in the second.
"You almost couldn't ask for a clearer story."
That is how Christian put it in Berkeley's write-up. He added that the study is not really about fractions or the SAT themselves.
How people use AI makes a big difference
In the second experiment, the AI group reported how they had used the tool: 61% asked the AI directly for answers, 27% asked only for hints or explanations, and 12% barely used it.
Those who asked for answers scored 65% on the test and skipped 13% of questions. Those who asked only for hints scored 76% and skipped 5%, close to the control group. The authors state plainly that this is a post-hoc comparison between subgroups and cannot be treated as causal evidence: how people choose to use AI may itself be tied to ability or personality.
The AI used in the study was deliberately set up to give complete answers wherever possible. Whether a tutoring-style or Socratic AI could preserve the benefit of help without eroding persistence is, in the authors' view, a promising direction, but none of the three experiments tested it directly. In the paper's conclusions, they call user-side measures such as "use it less" or "switch to hint mode" stopgaps, and argue that model developers should build users' long-term capabilities into their designs.
What it means for classrooms and parents in China
Schoolchildren in China have many ways to get AI help with homework: photo-based question search, homework apps and general-purpose chat assistants can all return a full solution in one step. This study tested exactly that "complete answer" mode, and after only 10 minutes.
Several limits need to be spelled out first. The participants were adults recruited online in the United States, and the tasks were basic fraction arithmetic and standardized reading, which is not the same as long-term, high-frequency homework use. The test came right after the AI was removed, and there is no data on how long the effect lasts. All three experiments also looked at a single exposure only; whether daily use would accumulate or people would adapt, the authors say, needs longitudinal research to answer.
What parents and teachers may find most useful is the gap of a few percentage points between asking for hints and asking for answers in the second experiment: with the same AI at hand, the way people asked led to different performance once it was removed. For now that is only a correlation, still waiting for someone to test it in a controlled experiment.
Sources: UC Berkeley press release, CocoLoop, fifth version of the arXiv paper; sample sizes, accuracy, skip rates and self-reported usage shares for the three experiments were checked against the paper's text.