Paper2Agent Turns Papers Into Research Tools

Paper2Agent packages a scientific paper’s code, methods and data into reusable tools that an AI assistant can apply to new data, according to research reported September 16 in Nature. Stanford computer scientist James Zou calls the system “a virtual author of the paper” available around the clock.

Zou and colleagues tested the system on 100 computational biology papers and converted 74 into working tool sets. The other papers had problems including missing code or data, or software that would not run. For papers it converted, Paper2Agent could run analyses from the published work and answer questions about results. It did so more accurately and more than twice as fast as the same agent working from the papers and accompanying code.

Peter Clark, a senior research director at the Allen Institute for Artificial Intelligence who did not help develop the system, says getting another researcher’s code to run can take substantial effort. He says he can spend half a day figuring out code shared on platforms such as GitHub. A general-purpose assistant such as Claude can help, but typically writes new code as needed. Paper2Agent instead packages a paper’s existing code into reusable tools, checks them against reference outputs, and writes step-by-step instructions for which tools to use and in what order. Clark says this standardizes how AI interacts with research.

Zou says his team has made “over 100 different paper-agents” from its own work, which have helped him find researchers to collaborate with. Paper-agents can also search for agents on similar topics and aim to alert their human authors to possible collaborations. Clark says Paper2Agent focuses on reusing methods and code, rather than answering general questions as a virtual author might. He also worries that agents made from millions of low-quality papers could pester scientists, recalling Microsoft Office’s Clippy assistant. For open-ended questions, Clark says he would still contact the researchers because he wants to learn things they left out of the paper.

Paper2Agent packages a scientific paper’s code, methods and data into reusable tools that an AI assistant can apply to new data, according to research reported September 16 in Nature. Stanford computer scientist James Zou calls the system “a virtual author of the paper” available around the clock.

Zou and colleagues tested the system on 100 computational biology papers and converted 74 into working tool sets. The other papers had problems including missing code or data, or software that would not run. For papers it converted, Paper2Agent could run analyses from the published work and answer questions about results. It did so more accurately and more than twice as fast as the same agent working from the papers and accompanying code.

Peter Clark, a senior research director at the Allen Institute for Artificial Intelligence who did not help develop the system, says getting another researcher’s code to run can take substantial effort. He says he can spend half a day figuring out code shared on platforms such as GitHub. A general-purpose assistant such as Claude can help, but typically writes new code as needed. Paper2Agent instead packages a paper’s existing code into reusable tools, checks them against reference outputs, and writes step-by-step instructions for which tools to use and in what order. Clark says this standardizes how AI interacts with research.

Zou says his team has made “over 100 different paper-agents” from its own work, which have helped him find researchers to collaborate with. Paper-agents can also search for agents on similar topics and aim to alert their human authors to possible collaborations. Clark says Paper2Agent focuses on reusing methods and code, rather than answering general questions as a virtual author might. He also worries that agents made from millions of low-quality papers could pester scientists, recalling Microsoft Office’s Clippy assistant. For open-ended questions, Clark says he would still contact the researchers because he wants to learn things they left out of the paper.