LiveBench 2.0 Coding
Higher is better
- What does it measure?
- Tests whether a model can fix real bugs in a large production codebase.
- Who evaluates it, and how?
- Each task is a bug that a real merged pull request fixed; the model gets the bug report and the repository frozen at a snapshot (29 tasks in the October 25, 2026 release). Hidden tests check the fix, and a fix that breaks existing behaviour scores 0.
- Reading the score
- Scores are shown as published by the source. Higher is better.
Top score89.7 / 100Claude Fable 5.1
Models tested22
Last updated2026.11.06
Model release date (newest on the right)
Best score so farHigher on the chart is better
Can AI carry out long development tasks autonomously by operating a computer itself?Terminal-Bench 4.0
Kimi K3.1 Pro🥇Kimi K3.1 Pro🥇Which AI do people prefer most for coding answers?Arena Coding
Gemini 4 Carbon🥇Gemini 4 Carbon🥇Can AI test and fix code itself until it completes the given task?LiveBench Agentic Coding v1
DeepSeek V4.1 Flash🥇DeepSeek V4.1 Flash🥇
Kimi K3.1 Pro🥇Kimi K3.1 Pro🥇Which AI do people prefer most for coding answers?Arena CodingGemini 4 Carbon🥇Gemini 4 Carbon🥇Can AI test and fix code itself until it completes the given task?LiveBench Agentic Coding v1DeepSeek V4.1 Flash🥇DeepSeek V4.1 Flash🥇- 🥇AnthropicClaude Fable 5.1Reasoning effort: Max89.7 / 100
- 🥈OpenAIGPT-6.1 TerraReasoning effort: Max89.2 / 100
- 🥉OpenAIGPT-6 AstraReasoning effort: Max87.5 / 100
- #4AnthropicClaude Opus 5.5Reasoning effort: Max87.1 / 100
- #5GoogleGemini 4 ArgonReasoning effort: High85.7 / 100
- #6OpenAIGPT-5.6 TerraReasoning effort: Max84.2 / 100
- #7MetaMuse Spark 1.3Reasoning effort: Max83.1 / 100
- #8OpenAIGPT-6.1 SolReasoning effort: Max82.1 / 100
- #9DeepSeekDeepSeek V4.1 Flash 1017Reasoning effort: Max80.4 / 100
- #10AnthropicClaude Fable 5.2Reasoning effort: Extra High79.2 / 100
- #11AnthropicClaude Haiku 5.5Reasoning effort: Max77.6 / 100
- #12AnthropicClaude Sonnet 5.5Reasoning effort: Max77.2 / 100
- #13SpaceXSIGrok 4.6Reasoning effort: Extra High76.7 / 100
- #14SpaceXSIGrok 4.7Reasoning effort: Extra High73.1 / 100
- #15Z.aiGLM 5.3 FlashReasoning effort: Max70.9 / 100
- #16OpenAIGPT-6 LunaReasoning effort: Max70.8 / 100
- #17Moonshot AIKimi K3.1 ProReasoning effort: Max67.8 / 100
- #18GoogleGemini 3.8 FlashReasoning effort: High65.7 / 100
- #19XiaomiMiMo-V2.6-ProReasoning effort: High64.5 / 100
- #20Moonshot AIKimi K3Reasoning effort: Max53.1 / 100
- #21Mistral AIMistral Large 4Reasoning effort: High44.4 / 100
- #22GoogleGemma 4 31BReasoning effort: Max20.3 / 100
Half of models ≤ 77.4 / 100Last updated 2026-11-11
The "harness" is the agent program the AI used to carry out the task. The same model can score very differently depending on its harness and reasoning effort.
Where the scores come from
- Run by the test’s team: scores the team that operates the test ran and published itself.