Ai2 launches Theorizer to propose hypotheses from papers

Open source illustration: hands joining branches to a shared node tree

The Allen Institute for AI (Ai2) on November 19 launched Theorizer on its research platform Asta. The agent pulls together findings from multiple papers and proposes explanations for observed phenomena. The code is open-sourced on GitHub at the same time.

Use starts with a question. After searching the literature, Theorizer first generates an extraction schema that specifies what to pull from each paper. In Ai2's example, if you ask how tumors grow, the schema might cover which genes a study looked at, how their activity was measured, and what happened when the genes were altered. The agent then compares findings across papers and generates a set of theories, each claim linked to its source papers and stated with its scope of applicability. Researchers can click back to the original text claim by claim to judge whether the evidence is sufficient and whether the experimental conditions are comparable. Claims that it considers newly proposed are flagged separately.

Three case studies, no benchmark scores

Ai2 says it spent close to a year refining the tool together with scientists. What it released this time is a handful of case studies, with no benchmark scores of any kind.

The first concerns lung adenocarcinoma. In roughly 20% to 30% of cases, no recognized driver mutation can be found even after comprehensive sequencing. Theorizer offered three directions: undiscovered genetic alterations that activate RAS signaling; other mechanisms that let cells divide while bypassing RAS; and some tumors taking on features of other cell types. Researcher Devarakonda reacted after reading the output:

"Wait a second, this is exactly what I think. And it took me 15 years to think like this."

The team later used another Ai2 tool, AutoDiscovery, to propose that disruption of the DREAM complex might be involved, and saw preliminary signs supporting a dependence on CDK2. Ai2 states that all of this still requires experimental validation.

Neuroscientist Lecoq asked about VIP interneurons. He had expected one or two candidate theories; Theorizer returned 50 in one go.

The third case is in Peru. Palacios and Juarez were working with the PISCO weather dataset, and one explanation Theorizer produced was later judged wrong by the two researchers. But in order to test it, they went back to check the November temperature peaks and traced the problem to the data source: the dew point record ends in 2016, while the minimum temperature record continues through 2020, a gap of 4 years. Some reconstructed values exceed 35°C and cluster in November. The two are still checking observations from independent stations to find out where the November peaks in the source records come from, and there is no conclusion yet.

The Asta line of work

Put Ai2's moves on Asta this year side by side and the division of labor among the tools is getting finer. In early October it open-sourced AstaBrief, an 8B-parameter model that writes retrieved literature into cited research reports, with an average generation time of 51.1 seconds at the time. AstaBrief takes the small-model, open-weights route; Ai2's reasoning is that for many scientists the best evidence sits in internal reports that cannot be sent to commercial AI labs. Theorizer goes one step further, from summarizing existing conclusions to proposing explanations that can be taken off and tested.

The barrier to entry is low. It can be used directly on the Asta website, where Ai2 covers the token costs up to a cap. To run it on your own machine, you can use the released library and pay token fees according to actual usage. The release page does not say which license the code uses, so the GitHub repository is the authority. Ai2 has also scheduled an online workshop for scientists at 10 a.m. Pacific Time on December 10.

Ai2 lists two limitations itself: Theorizer may miss relevant studies, and it may misjudge originality, so a theory that looks new within the batch of papers it analyzed may already have been proposed elsewhere. Ai2 has not given statistics on how many of the claims flagged as new fall into that category, so researchers still need to check against a broader body of literature.

Sources: CocoLoop, Ai2 official blog, the Asta platform, the Ai2 GitHub repository; verified the Theorizer workflow, the proportions and years cited in each case study, AstaBrief's average generation time, and the token-cost rules for the web version.