Terminal-Bench 4.0
- What does it measure?
- Whether an AI agent can finish long jobs inside a terminal (command line) with no human help: 66 tasks across software, machine learning, science, operations, security, hardware and media. The tasks are longer and harder than in 2.1, so it still separates the newest models now that 2.1 has top scores bunched around 90%.
- Who evaluates it, and how?
- Terminal-Bench 4.0 from Stanford and the Laude Institute. Each task runs in an isolated container where the AI works only by issuing shell commands (up to 8 hours per task); afterwards a hidden test suite grades it, and the task counts only if every test passes. Version 4.0 recalibrated compute and time limits and removed eight tasks that had become saturated or had quality problems. The values here are Artificial Analysis's own runs with the mini-swe-agent harness, averaged over three runs per task.
- Reading the score
- 0–100%. Even the top models are around 60%, and most small models score in single digits, so gaps between models are wide. Scores depend on the agent harness used with the model, so check that the run conditions match before comparing with Terminal-Bench numbers from vendors or the official leaderboard.
Rank
Top score67.2%
Models tested84
Scoring basisHigher is better
Last updated2026.11.21
Model release date
Which AI do people prefer most for coding answers?Arena Coding
Gemini 4 Carbon🥇
Gemini 4 Carbon🥇Arena Coding
Gemini 4 Carbon🥇Can AI test and fix code itself until it completes the given task?LiveBench Agentic Coding v1
DeepSeek V4.1 Flash🥇
DeepSeek V4.1 Flash🥇LiveBench Agentic Coding v1
DeepSeek V4.1 Flash🥇
Can AI carry out long development tasks autonomously by operating a computer itself?
Terminal-Bench 4.0
- 🥇Moonshot AIKimi K3.1 Pro67.2%1532—
- 🥈AnthropicClaude Sonnet 5.563.6%151739.3
- 🥉AnthropicClaude Opus 5.559.6%154865.3
- 🥉OpenAIGPT-6 Astra59.6%154857.3
- #5GoogleGemini 4 Argon57.1%1552—
- #6OpenAIGPT-6.1 Sol56.1%154556.8
- #7OpenAIGPT-6.1 Terra55.6%1541—
- #8AnthropicClaude Fable 5.155.1%153366.1
- #9MetaMuse Wonder52.0%1550—
- #10AnthropicClaude Opus 549.0%152865.2
- #11OpenAIGPT-6 Sol43.9%151952.9
- #12AnthropicClaude Fable 542.4%155162.2
- #13Z.aiGLM 5.341.9%152460.9
- #14OpenAIGPT-5.6 Sol39.9%153356.2
- #15OpenAIGPT-5.6 Terra35.4%151855.0
- #16XiaomiMiMo-V2.6-Pro34.8%1539—
- #17DeepSeekDeepSeek V4.1 Flash 101733.8%——
- #18MetaMuse Spark 1.333.3%153564.1
- #18StepfunStep-533.3%1504—
- #20AnthropicClaude Haiku 5.532.8%148751.4
- #20Z.aiGLM 5.3 Flash32.8%152656.8
- #22DeepSeekDeepSeek V4.1 Flash26.8%152577.3
- #22Mistral AIMistral Large 426.8%148657.2
- #24SpaceXSIGrok 4.725.8%149154.0
- #24MiniMaxMiniMax M3-1 Flash25.8%——
- #26AlibabaQwen3.8 Flash25.3%—61.6
- #27XiaomiMiMo-V2.6-Flash22.7%1510—
- #28AnthropicClaude Opus 4.821.7%152850.5
- #29SpaceXSIGrok 4.621.2%150857.0
- #30GoogleGemini 3.8 Flash19.7%153054.2
- #31OpenAIGPT-5.514.6%151054.0
- #32AnthropicClaude Sonnet 514.1%151959.4
- #32DeepSeekDeepSeek V4 Pro14.1%150142.6
- #34GoogleGemini 3.7 Flash13.6%152158.3
- #35OpenAIGPT-6 Luna12.6%150451.2
- #35Moonshot AIKimi K312.6%153762.2
- #37DeepSeekDeepSeek V4 Flash12.1%148437.6
- #38OpenAIGPT-5.6 Luna11.6%150348.4
- #39SpaceXSIGrok 4.510.6%151456.5
- #40GoogleGemini 3.6 Flash7.1%151843.4
- #40MetaMuse Spark 1.27.1%154057.6
- #42GoogleGemini 3.5 Flash6.6%150649.0
- #43MetaMuse Spark 1.16.1%153358.5
- #44GoogleGemini 3.1 Pro4.0%152144.1
- #45AnthropicClaude Sonnet 4.63.0%152842.6
- #46Z.aiGLM-5.12.0%1512—
- #46OpenAIGPT-5.4 Mini2.0%149541.7
- #46MiniMaxMiniMax M32.0%149240.7
- #49AlibabaQwen3.7 Max1.5%152443.6
- #50GoogleGemini 3.5 Flash-Lite1.0%150245.3
- #50Z.aiGLM 5.21.0%151651.8
- #50Moonshot AIKimi K2.7 Code1.0%—45.7
- #50AlibabaQwen3.7 Plus1.0%1500—
- #50UpstageSolar Mini 41.0%——
- #55GoogleGemini 3.1 Flash Lite0.5%1458—
- #55OpenAIGPT-5.4 Nano0.5%146046.8
- #55TencentHy30.5%1452—
- #55Moonshot AIKimi K2.60.5%151646.9
- #55NVIDIANemotron 3 Ultra0.5%147438.7
- #55CohereNorth Mini Code0.5%——
- #55InclusionAIRing-2.6-1T0.5%——
- #55Arcee AITrinity Large Thinking0.5%1412—
- #63SK TelecomA.X K20.0%——
- #63AnthropicClaude Haiku 4.50.0%1482—
- #63AnthropicClaude Sonnet 4.50.0%1512—
- #63GoogleGemini 2.5 Pro0.0%1464—
- #63GoogleGemma 4 31B0.0%1502—
- #63OpenAIGPT OSS 120B0.0%1389—
- #63OpenAIGPT-5 Mini0.0%1432—
- #63SpaceXSIGrok 4.30.0%148718.5
- #63MetaLlama 4 Maverick0.0%1372—
- #63MetaLlama 4 Scout0.0%1362—
- #63MeituanLongCat 2.00.0%——
- #63XiaomiMiMo V2.50.0%1491—
- #63XiaomiMiMo V2.5 Pro0.0%1520—
- #63MiniMaxMiniMax M2.70.0%1480—
- #63Mistral AIMistral Medium 3.50.0%1478—
- #63Mistral AIMistral Small 40.0%——
- #63NVIDIANemotron 3 Nano 30B A3B0.0%1361—
- #63NVIDIANemotron 3 Super0.0%1407—
- #63AlibabaQwen3.5 397B A17B0.0%1491—
- #63AlibabaQwen3.6 35B A3B0.0%——
- #63UpstageSolar Pro 30.0%——
- #63UpstageSolar Pro 40.0%1461—
Half of models ≤ 6.3%
49 of the 64 AIs released in the last 60 days have no score here yet · Gemini 4 Neon, GPT-6.2 Astra, Microsoft-Decision-1.1 and 46 more
This score comes from a fixed, predefined evaluation, so it may differ from what you get with your own question.
Scores labelled 'Self-reported' were published by the AI maker itself, not measured by a third party.