SciCode

Name: SciCode
Creator: aib vote
License: https://www.gnu.org/licenses/gpl-3.0.html

About This Benchmark

A benchmark of research-level scientific coding problems in math, physics, chemistry, and biology. Evaluates code generation at the level of real scientific research. Score is accuracy (%).

Source: Artificial Analysis

Rank	Model
#1	Google Gemini 3.1 Pro	58.9%
#2	OpenAI GPT-5.4	56.6%
#3	Anthropic Claude Opus 4.7	54.5%
#4	Anthropic Claude Opus 4.6	51.9%
#5	Meta Muse Spark	51.5%
#6	Google Gemini 3 Flash	50.6%
#7	DeepSeek DeepSeek V4 Pro	50.0%
#8	Anthropic Claude Opus 4.5	49.5%
#9	Moonshot AI Kimi K2.5	49.0%
#10	MiniMax MiniMax M2.7	47.0%
#11	Z.ai GLM-5	46.2%
#12	Grok Grok 4.20 (Reasoning)	45.6%
#13	DeepSeek DeepSeek V4 Flash	44.9%
#14	Anthropic Claude Sonnet 4.5	44.7%
#15	OpenAI GPT-5.4 Mini	44.2%
#16	Grok Grok 4.1 Fast (Reasoning)	44.2%
#17	Anthropic Claude Sonnet 4.6	44.1%
#18	Z.ai GLM-5.1	43.8%
#19	Google Gemma 4 31B	43.4%
#20	Anthropic Claude Haiku 4.5	43.3%
#21	Google Gemini 2.5 Pro	42.8%
#22	MiniMax MiniMax M2.5	42.6%
#23	Xiaomi MiMo-V2-Pro	42.5%
#24	Google Gemini 3.1 Flash Lite	41.9%
#25	Alibaba Qwen3.5 397B A17B	41.1%
#26	Anthropic Claude Opus 4.1	40.9%
#27	Alibaba Qwen3.6 Plus	40.7%
#28	Anthropic Claude Sonnet 4	40.0%
#29	Anthropic Claude Opus 4	39.8%
#30	Google Gemini 2.5 Flash	39.4%
#31	DeepSeek DeepSeek V3.2	38.9%
#32	OpenAI GPT-5.4 Nano	38.4%
#33	OpenAI GPT-4.1	38.1%
#34	Mistral AI Mistral Small 4	38.0%
#35	OpenAI GPT-5	37.8%
#36	Baidu ERNIE 5.0 Thinking	37.5%
#37	OpenAI GPT-5 Mini	36.9%
#38	Arcee AI Trinity Large Thinking	36.1%
#39	OpenAI GPT OSS 120B	36.0%
#40	NVIDIA Nemotron 3 Super	36.0%
#41	OpenAI GPT-5 Nano	33.8%
#42	Meta Llama 4 Maverick	33.1%
#43	Grok Grok 4.20	32.8%
#44	Baidu ERNIE 4.5 300B A47B	31.5%
#45	Grok Grok 4.1 Fast	29.6%
#46	Google Gemini 2.5 Flash Lite	28.5%
#47	Meituan Longcat Flash Chat	28.4%
#48	Amazon Nova 2 Lite	24.0%
#49	Meta Llama 4 Scout	17.0%