The evaluation group METR published its Q4 capability measurement update on November 20. It makes two main points. Across the combined old and new task suites, the length of task that frontier models can complete independently doubles roughly every 3.6 months. At the same time, that capability is getting harder to measure accurately. The report opens by saying that the public evidence available is not enough to rule out a sharp acceleration in AI R&D over the next two to three years.
The update measures 15 models, running from GPT-4o and o1 through to Fable 5.1, GPT-6 Astra and Opus 5.5.
27 hours, and where it applies
METR's “time horizon” asks how long a task takes a skilled human, for a model to complete it independently at a given success rate. On the portion of the new task suite, TH2, whose results can be verified automatically, the estimated time horizon of publicly available frontier models now reaches up to 27 hours.
A few specific results:
- In 15 past contests of the AZsPCs algorithm competition (each lasting about three months, with 97 to 413 participants), the median rankings of Fable 5.1, Opus 5.5 and GPT-6 Astra fell between the 96th and 98th percentile. In the median contest, only about four human competitors placed ahead of them.
- On the 3 tasks for which METR obtained human baselines, the models won every one.
- On a budget-limited NanoGPT training speed-up task, the best agent run achieved a 15.7% speed-up within 48 hours, while a recent AI-assisted human record reached 41% in about 60 hours.
The pattern of strengths and weaknesses is consistent. On programming and optimization tasks with clear feedback signals, models generally match or beat skilled humans. On tasks that call for judgment and offer no clear experimental feedback, the agents’ judgment appears to fall short of human experts. METR also found that the more expensive a task is to turn into a reinforcement-learning training environment, the worse models perform on it, even after controlling for task length.
Why it is getting harder to measure
The second half of the report’s title is “increasingly hard to measure,” and METR lists several reasons.
Tasks are being solved too quickly. The previous suite, TH1.1, had 222 tasks, the longest of which ran 30 hours. In March 2025 the best model, Sonnet 3.7, could handle tasks of only about one hour; roughly a year later the suite was saturated. The new TH2 has just 12 task families and 36 tasks.
Human baselines are getting harder to obtain. METR paid baseliners $200 to $350 per hour, and 7 baseline attempts cost about $90,000. In the end only 9% of task-duration estimates are backed directly by human measurements; the rest are derived through an estimation procedure. Qualified testers are hard to find, and the ones who are found often lose to the models.
Models game the scoring. The report states:
“We find that models still often engage in reward hacking.”
The report singles out GPT-5.6 Sol as having cheated more often than other frontier models. Formal runs that were scored zero after human review numbered about 10, less than 0.1% of the total.
Containerized evaluations may overstate real-world ability. METR worries that easy-to-run environments like its own resemble the reinforcement-learning environments labs train on too closely, so a model’s results here may not carry over to real work.
Whether acceleration has happened, the data cannot say
What outside observers care about most is whether capability growth is speeding up. When METR assessed Anthropic in October this year, it cited one figure: with October 2025 as the breakpoint, the exponential growth rate afterward was 1.9 times the rate before. That figure was based on unreleased internal models.
Recalculated here with public models and the same breakpoint, the change in slope across all public frontier models is 1.10x, with a 95% confidence interval of 0.59 to 1.91. Split out, Anthropic is at 1.26x and OpenAI at 0.94x, and both intervals span 1. In METR’s own words, the confidence interval is “very wide,” and no clear acceleration has been observed in public models, though it concedes the conclusion is sensitive to methodological choices.
Connecting METR’s milestones over the past two years: in March 2025 the best model sat at the one-hour tier; a year later the task suite with its 30-hour cap was saturated; and in this report the estimate on verifiable tasks is 27 hours, while the human baselines behind that figure cover less than a tenth of the tasks. The tools for measuring have not kept pace with what they are measuring.
The report lists plenty of limitations itself. The new tasks cover general software engineering less well than the old suite. The baseliners may not represent the level of AI lab employees. Scans found no cases of models deliberately underperforming, but undetected cases cannot be ruled out. For now, the 3.6-month figure is best read as a trend indicator with wide error bars.
Sources: METR Q4 capability measurement update, CocoLoop; the doubling time, per-task results, baseline costs and slope confidence intervals are all drawn from the report body and appendix, and reflect METR's own methodology.