Vals AI Ranks AI Models on Science Research Ability, Strengths Vary by Model

Vals AI announced on the 16th (local time) the Vals Science Index, which scores AI scientific research ability as a single number for the first time. The index evaluates biological data analysis, principle discovery through experiments, and actual scientific research tasks, combining three existing science benchmarks with equal weighting: Anthropic's BioMysteryBench, Vals AI's MysteryMechanism, and TerminalBench Science.

Twenty-one AI models took part in the evaluation. OpenAI's GPT-6 Astra took first place with 65.09%, followed by GPT-6.1 Sol at 59.63% and Claude Sonnet 5.5 at 59.12%. Claude Opus 5.5 ranked fourth at 58.65%, and Google's Gemini 4 Argon placed fifth at 55.36%. Astra finished more than 5 percentage points ahead of second place to take the overall lead.

The detailed results show that each model has different strengths. Astra ranked first in hidden-principle discovery and actual scientific research tasks but third in biological data analysis, a category Claude Sonnet 5.5 led. By field, Astra scored highest in life sciences and mathematics, Claude Sonnet 5.5 in physics and engineering, and GPT-6.1 Sol in earth sciences.

Rankings differed on Vals AI's existing comprehensive index, which evaluates real task performance in finance, coding, law, and tax, weighted by industry share of GDP. Gemini 4 Argon ranked first there at 68.90%, while Astra, first on the science index, ranked seventh at 63.13%. Vals AI said this shows it is hard to identify a top model with a single composite score.

Vals AI is a US-based AI evaluation startup founded in 2024. In August it raised $40 million in a round led by Andreessen Horowitz (a16z), valuing the company at $400 million. The company says OpenAI, Anthropic, Google, and Meta cite its evaluation results when announcing model performance, and its revenue through August this year reached eight times last year's annual revenue.

The index was created to address a limitation of existing science benchmarks, which the company says have focused on recalling known knowledge and checking correct answers. The company said it aims to measure data analysis and hypothesis testing needed for real scientific research, as well as the ability to infer principles from observation alone. However, it did not develop a new evaluation method this time, and it explained that the index is simply the average of the three benchmarks and does not mean AI independently completes scientific discovery.

Vals AI announced on the 16th (local time) the Vals Science Index, which scores AI scientific research ability as a single number for the first time. The index evaluates biological data analysis, principle discovery through experiments, and actual scientific research tasks, combining three existing science benchmarks with equal weighting: Anthropic's BioMysteryBench, Vals AI's MysteryMechanism, and TerminalBench Science.

Twenty-one AI models took part in the evaluation. OpenAI's GPT-6 Astra took first place with 65.09%, followed by GPT-6.1 Sol at 59.63% and Claude Sonnet 5.5 at 59.12%. Claude Opus 5.5 ranked fourth at 58.65%, and Google's Gemini 4 Argon placed fifth at 55.36%. Astra finished more than 5 percentage points ahead of second place to take the overall lead.

The detailed results show that each model has different strengths. Astra ranked first in hidden-principle discovery and actual scientific research tasks but third in biological data analysis, a category Claude Sonnet 5.5 led. By field, Astra scored highest in life sciences and mathematics, Claude Sonnet 5.5 in physics and engineering, and GPT-6.1 Sol in earth sciences.

Rankings differed on Vals AI's existing comprehensive index, which evaluates real task performance in finance, coding, law, and tax, weighted by industry share of GDP. Gemini 4 Argon ranked first there at 68.90%, while Astra, first on the science index, ranked seventh at 63.13%. Vals AI said this shows it is hard to identify a top model with a single composite score.

Vals AI is a US-based AI evaluation startup founded in 2024. In August it raised $40 million in a round led by Andreessen Horowitz (a16z), valuing the company at $400 million. The company says OpenAI, Anthropic, Google, and Meta cite its evaluation results when announcing model performance, and its revenue through August this year reached eight times last year's annual revenue.

The index was created to address a limitation of existing science benchmarks, which the company says have focused on recalling known knowledge and checking correct answers. The company said it aims to measure data analysis and hypothesis testing needed for real scientific research, as well as the ability to infer principles from observation alone. However, it did not develop a new evaluation method this time, and it explained that the index is simply the average of the three benchmarks and does not mean AI independently completes scientific discovery.