Our Story

TerminalBench Hard

About This Benchmark

A hard-level benchmark evaluating AI agents' ability to execute complex shell commands, file operations, and system tasks in a real terminal environment. Score is success rate (%).

Source: Artificial Analysis
RankModel
#1

OpenAI

GPT-5.4

57.6%
#2

Google

Gemini 3.1 Pro

53.8%
#3

Anthropic

Claude Opus 4.7

51.5%
#4

Anthropic

Claude Opus 4.5

47.0%
#5

Anthropic

Claude Opus 4.6

46.2%
#6

DeepSeek

DeepSeek V4 Pro

46.2%
#7

Meta

Muse Spark

45.5%
#8

Alibaba

Qwen3.6 Plus

43.9%
#9

Z.ai

GLM-5

43.2%
#10

Z.ai

GLM-5.1

43.2%
#11

Anthropic

Claude Sonnet 4.6

42.4%
#12

Xiaomi

MiMo-V2-Pro

40.9%
#13

MiniMax

MiniMax M2.7

39.4%
#14

Google

Gemini 3 Flash

38.6%
#15

Grok

Grok 4.20 (Reasoning)

37.9%
#16

Google

Gemma 4 31B

36.4%
#17

Anthropic

Claude Sonnet 4.5

35.6%
#18

DeepSeek

DeepSeek V3.2

35.6%
#19

DeepSeek

DeepSeek V4 Flash

35.6%
#20

Alibaba

Qwen3.5 397B A17B

35.6%
#21

Moonshot AI

Kimi K2.5

34.8%
#22

MiniMax

MiniMax M2.5

34.8%
#23

Anthropic

Claude Opus 4.1

34.3%
#24

OpenAI

GPT-5.4 Mini

34.1%
#25

OpenAI

GPT-5.4 Nano

33.3%
#26

Anthropic

Claude Opus 4

31.1%
#27

Anthropic

Claude Sonnet 4

31.1%
#28

NVIDIA

Nemotron 3 Super

28.8%
#29

Anthropic

Claude Haiku 4.5

27.3%
#30

Google

Gemini 2.5 Pro

26.5%
#31

Baidu

ERNIE 5.0 Thinking

25.0%
#32

Google

Gemini 3.1 Flash Lite

24.2%
#33

Grok

Grok 4.1 Fast (Reasoning)

24.2%
#34

Arcee AI

Trinity Large Thinking

22.7%
#35

OpenAI

GPT-5 Nano

17.4%
#36

Mistral AI

Mistral Small 4

17.4%
#37

Grok

Grok 4.20

16.7%
#38

OpenAI

GPT-5 Mini

14.4%
#39

Grok

Grok 4.1 Fast

14.4%
#40

Google

Gemini 2.5 Flash

13.6%
#41

OpenAI

GPT-4.1

13.6%
#42

OpenAI

GPT-5

12.9%
#43

Meituan

Longcat Flash Chat

10.6%
#44

Google

Gemini 2.5 Flash Lite

7.6%
#45

Meta

Llama 4 Maverick

6.8%
#46

Amazon

Nova 2 Lite

6.8%
#47

Baidu

ERNIE 4.5 300B A47B

6.1%
#48

OpenAI

GPT OSS 120B

5.3%
#49

Meta

Llama 4 Scout

1.5%