AI Dispute Raises Concerns About Scientific Trust

A dispute among mathematicians has raised concerns that AI could weaken trust in scientific research. Last month, OpenAI said its AI agents solved a major open problem involving the Navier–Stokes equations in 88 hours. The organization that awards a prize for solving the problem said it had “apparently been settled,” but that its review would take time. Hours before OpenAI’s announcement, New York University mathematician Tristan Buckmaster questioned whether his and Levent Alpöge’s work, stored in OpenAI’s Codex coding tool, could have influenced the company’s result. Buckmaster said he was not accusing anyone of wrongdoing. OpenAI said no one at the company had seen the other team’s work before publication. It first said anonymized product-use data might have helped improve its models, though that was unlikely; days later, it said Buckmaster’s Codex prompts from the previous two months could not have influenced the system. The article says what happened remains unclear.

Scientists also use AI to process vast datasets, such as those collected by telescopes and particle accelerators. Google DeepMind’s AlphaFold predicted 200 million protein folds in one year, work that would have taken hundreds of millions of years of laboratory experiments. The company made the results freely available, and DeepMind’s head and the project’s lead scientist received the Nobel Prize in Chemistry. DeepMind later said its model had generated more than two million new crystal structures, calling the output “nearly 800 years worth of knowledge”; 380,000 were judged promising for laboratory work. University of California chemists Anthony Cheetham and Ram Seshadri examined a random sample and said they had not found strikingly novel compounds, describing many as minor changes to known materials. DeepMind said it tested the structures for novelty and argued that changing one atom can produce useful materials.

Concerns also involve AI-written papers, fabricated citations and peer review. In a search of more than one million biomedical papers, Ghent University’s Dmitry Kobak found that nearly 90% had AI fingerprints by December last year; he warned that biases in language models could spread through scientific literature. A Stanford-led analysis of referee reports submitted to four major computer science conferences found that up to 17% of the text had been substantially modified by a language model. Cornell researchers separately found around 147,000 pieces of phantom research cited last year. The article says distorted research could send scientists down unproductive paths, waste public money and delay or prevent useful discoveries, including drugs and green technologies.

Some publishers have introduced checks. Elsevier’s research-integrity team uses around 100 signals to flag papers for closer human scrutiny. Nature journals ban AI-generated images and tell referees not to upload manuscripts to AI systems; arXiv bans researchers for a year for submitting invented references. DeepMind science lead Pushmeet Kohli says researchers should remain in control and accept responsibility for AI agents’ work. Vienna BioCenter’s Andrea Pauli used AI in research that identified a protein on zebrafish eggs pointing to a lock-and-key mechanism for sperm entry. Basecamp Research co-founder Oliver Vince says his company is pursuing a technically feasible path to curing cancer, while acknowledging it may not work. The article’s central question is whether AI will help science advance or erode trust in its foundations.

A dispute among mathematicians has raised concerns that AI could weaken trust in scientific research. Last month, OpenAI said its AI agents solved a major open problem involving the Navier–Stokes equations in 88 hours. The organization that awards a prize for solving the problem said it had “apparently been settled,” but that its review would take time. Hours before OpenAI’s announcement, New York University mathematician Tristan Buckmaster questioned whether his and Levent Alpöge’s work, stored in OpenAI’s Codex coding tool, could have influenced the company’s result. Buckmaster said he was not accusing anyone of wrongdoing. OpenAI said no one at the company had seen the other team’s work before publication. It first said anonymized product-use data might have helped improve its models, though that was unlikely; days later, it said Buckmaster’s Codex prompts from the previous two months could not have influenced the system. The article says what happened remains unclear.

Scientists also use AI to process vast datasets, such as those collected by telescopes and particle accelerators. Google DeepMind’s AlphaFold predicted 200 million protein folds in one year, work that would have taken hundreds of millions of years of laboratory experiments. The company made the results freely available, and DeepMind’s head and the project’s lead scientist received the Nobel Prize in Chemistry. DeepMind later said its model had generated more than two million new crystal structures, calling the output “nearly 800 years worth of knowledge”; 380,000 were judged promising for laboratory work. University of California chemists Anthony Cheetham and Ram Seshadri examined a random sample and said they had not found strikingly novel compounds, describing many as minor changes to known materials. DeepMind said it tested the structures for novelty and argued that changing one atom can produce useful materials.

Concerns also involve AI-written papers, fabricated citations and peer review. In a search of more than one million biomedical papers, Ghent University’s Dmitry Kobak found that nearly 90% had AI fingerprints by December last year; he warned that biases in language models could spread through scientific literature. A Stanford-led analysis of referee reports submitted to four major computer science conferences found that up to 17% of the text had been substantially modified by a language model. Cornell researchers separately found around 147,000 pieces of phantom research cited last year. The article says distorted research could send scientists down unproductive paths, waste public money and delay or prevent useful discoveries, including drugs and green technologies.

Some publishers have introduced checks. Elsevier’s research-integrity team uses around 100 signals to flag papers for closer human scrutiny. Nature journals ban AI-generated images and tell referees not to upload manuscripts to AI systems; arXiv bans researchers for a year for submitting invented references. DeepMind science lead Pushmeet Kohli says researchers should remain in control and accept responsibility for AI agents’ work. Vienna BioCenter’s Andrea Pauli used AI in research that identified a protein on zebrafish eggs pointing to a lock-and-key mechanism for sperm entry. Basecamp Research co-founder Oliver Vince says his company is pursuing a technically feasible path to curing cancer, while acknowledging it may not work. The article’s central question is whether AI will help science advance or erode trust in its foundations.