TerminalBench Hard

Name: TerminalBench Hard
Creator: aib vote
License: https://www.gnu.org/licenses/gpl-3.0.html

このベンチマークについて

実際のターミナル環境でのシェルコマンド実行やファイル操作能力を評価するHard難易度ベンチマーク。スコアは成功率(%)です。

順位	モデル
#1	OpenAI GPT-5.4	57.6%
#2	Google Gemini 3.1 Pro	53.8%
#3	Anthropic Claude Opus 4.7	51.5%
#4	Anthropic Claude Opus 4.5	47.0%
#5	Anthropic Claude Opus 4.6	46.2%
#6	DeepSeek DeepSeek V4 Pro	46.2%
#7	Meta Muse Spark	45.5%
#8	Alibaba Qwen3.6 Plus	43.9%
#9	Z.ai GLM-5	43.2%
#10	Z.ai GLM-5.1	43.2%
#11	Anthropic Claude Sonnet 4.6	42.4%
#12	Xiaomi MiMo-V2-Pro	40.9%
#13	MiniMax MiniMax M2.7	39.4%
#14	Google Gemini 3 Flash	38.6%
#15	Grok Grok 4.20 (Reasoning)	37.9%
#16	Google Gemma 4 31B	36.4%
#17	Anthropic Claude Sonnet 4.5	35.6%
#18	DeepSeek DeepSeek V3.2	35.6%
#19	DeepSeek DeepSeek V4 Flash	35.6%
#20	Alibaba Qwen3.5 397B A17B	35.6%
#21	Moonshot AI Kimi K2.5	34.8%
#22	MiniMax MiniMax M2.5	34.8%
#23	Anthropic Claude Opus 4.1	34.3%
#24	OpenAI GPT-5.4 Mini	34.1%
#25	OpenAI GPT-5.4 Nano	33.3%
#26	Anthropic Claude Opus 4	31.1%
#27	Anthropic Claude Sonnet 4	31.1%
#28	NVIDIA Nemotron 3 Super	28.8%
#29	Anthropic Claude Haiku 4.5	27.3%
#30	Google Gemini 2.5 Pro	26.5%
#31	Baidu ERNIE 5.0 Thinking	25.0%
#32	Google Gemini 3.1 Flash Lite	24.2%
#33	Grok Grok 4.1 Fast (Reasoning)	24.2%
#34	Arcee AI Trinity Large Thinking	22.7%
#35	OpenAI GPT-5 Nano	17.4%
#36	Mistral AI Mistral Small 4	17.4%
#37	Grok Grok 4.20	16.7%
#38	OpenAI GPT-5 Mini	14.4%
#39	Grok Grok 4.1 Fast	14.4%
#40	Google Gemini 2.5 Flash	13.6%
#41	OpenAI GPT-4.1	13.6%
#42	OpenAI GPT-5	12.9%
#43	Meituan Longcat Flash Chat	10.6%
#44	Google Gemini 2.5 Flash Lite	7.6%
#45	Meta Llama 4 Maverick	6.8%
#46	Amazon Nova 2 Lite	6.8%
#47	Baidu ERNIE 4.5 300B A47B	6.1%
#48	OpenAI GPT OSS 120B	5.3%
#49	Meta Llama 4 Scout	1.5%