Q1 How strong is it across many tests combined? AA Intelligence Index · Bottom 26%#88 Q1 How strong is it across many tests combined? AA Intelligence Index · Bottom 26%#88 Q2 Which AI gives the answers people prefer most? Arena Text · Bottom 17%#82 Q3 Does it have broad knowledge across expert fields? MMLU-Pro · Top 40%#11
Q1 #88 of 117 18.9 Reasoning effort: Thinking Q2 1402 Q3 84.2% By effort None 16.6 Thinking 18.9
About AA Intelligence Index More about AA Intelligence Index
Q1 Can it handle customer service by the rules? τ³-Banking · Bottom 42%#47 Q1 Can it handle customer service by the rules? τ³-Banking · Bottom 42%#47 Q2 Can it handle customer service by the rules? TAU2 · Bottom 22%#63 Q3 Can it finish long tasks that take people hours? METR Time Horizon · Bottom 20%#9 Q1 #47 of 79 16.7% Reasoning effort: Thinking Q2 64.6% Q3 1h 15m Q4 33.1% Q5 43.9% About τ³-Banking More about τ³-Banking
Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Bottom 30%#83 Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Bottom 30%#83 Q2 Which AI gives the business answers people prefer most? Arena Business, Management & Finance · Bottom 13%#86 Q3 Can it read a contract hundreds of pages long and find exactly what's needed? CorpFin v2 · Bottom 34%#43 Q1 #83 of 116 70.3% Reasoning effort: Thinking Q2 1381 Q3 61.2% Q4 35.0% Q5 46.9% Q6 44.5% By effort None 44.0% Thinking 70.3%
About Long Context Reasoning More about Long Context Reasoning
Q1 Can it carry out multi-step web research? Deep Research Bench · Bottom 36%#12 Q1 Can it carry out multi-step web research? Deep Research Bench · Bottom 36%#12 Q2 Does it avoid making things up when summarizing? Vectara Hallucination Rate · Bottom 48%#20 Q3 Does it avoid making things up when summarizing? Vectara Factual Consistency · Bottom 48%#20 Q4 How well can it predict what will happen? ForecastBench Brier Index · Bottom 34%#19
Q1 #12 of 17 46.6% Reasoning effort: Budget 2K Q2 10.3% Q3 89.7% Q4 58.9 About Deep Research Bench More about Deep Research Bench
Q1 Which AI gives the coding answers people prefer most? Arena Coding · Bottom 25%#74 Q1 Which AI gives the coding answers people prefer most? Arena Coding · Bottom 25%#74 Q2 Can it find better answers to optimization problems with no single right answer? ALE-Bench · Bottom 22%#59 Q3 Can it operate a computer itself to finish development jobs on its own? Terminal-Bench 2.1 · Bottom 21%#65 Q1 #74 of 97 1474 Q2 655 Q3 36.3% Q4 31.1% Q5 37.6 Q6 65.5% Q7 40.0% Q8 46.1% Q9 4.9% Q10 61.3% Q11 1445 About Arena Coding More about Arena Coding
Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Bottom 22%#77 Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Bottom 22%#77 Q2 Which AI do people prefer most in long conversations? Arena Multi-turn · Bottom 22%#77 Q3 Does it follow tricky rules like format and length without missing any? IFBench · Bottom 24%#63 Q1 #77 of 97 1398 Q2 1421 Q3 54.7% Q4 1396 Q5 1416 Q6 8.14 / 10 Q7 72.4% Q8 1389 About Arena Writing, Literature & Language More about Arena Writing, Literature & Language
Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Bottom 12%#104 Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Bottom 12%#104 Q2 Can it solve research-level physics problems? CritPt · Bottom 12%#70 Q3 Which AI gives the math answers people prefer most? Arena Math · Bottom 15%#81 Q1 #104 of 117 10.7% Reasoning effort: Thinking Q2 0.3% Q3 1404 Q4 77.7% Q5 71.1% Q6 74.3 Q7 74.3% Q8 1416 Q9 1413 Q10 99.1% Q11 84.4% Q12 4.1% Q13 0.0% Q14 40.0% Q15 5.9% Q16 45.5% Q17 3.1% Q18 77.1% Q19 29.0 By effort None 4.3% Thinking 10.7%
About Humanity's Last Exam More about Humanity's Last Exam
Q1 Which AI gives the photo and chart answers people prefer most? Arena Vision · Bottom 7%#55 Q1 Which AI gives the photo and chart answers people prefer most? Arena Vision · Bottom 7%#55 Q2 Can it predict how objects will move just from a picture? VPCT · Bottom 20%#9 Q3 Can it tell where a photo was taken from a single image? GeoBench · Bottom 15%#7 Q4 Can it understand a space from photos taken at different angles? MindCube ·
Q1 #55 of 58 1191 Q2 34.0% Q3 37.0% Q4 44.8% About Arena Vision More about Arena Vision
Q1 Which AI do people prefer most when asking in Korean? Arena Korean · Bottom 16%#68 Q1 Which AI do people prefer most when asking in Korean? Arena Korean · Bottom 16%#68 Q2 Which AI do people prefer most when asking in Japanese? Arena Japanese · Bottom 16%#64 Q3 Which AI do people prefer most on tough questions? Arena Hard Prompts · Bottom 18%#81 Q1 #68 of 79 1338 Q2 1342 Q3 1432 Q4 1433 Q5 1416 Q6 1406 Q7 35.0% Q8 17.9% Q9 56.1% Q10 28.6% Q11 20.4% Q12 35.0% Q13 90.6% Q14 90.0% Q15 91.3% Q16 97.8% Q17 81.5% Q18 28.3% Q19 79.2% Q20 89.3% Q21 74.6% Q22 6.0% Q23 24.6% About Arena Korean More about Arena Korean
