Q1 Which AI gives the answers people prefer most? Arena Text · Bottom 14%#94 Q1 Which AI gives the answers people prefer most? Arena Text · Bottom 14%#94 Q2 Does it have broad knowledge across expert fields? MMLU-Pro · Top 40%#11 Q3 How high was its intelligence index on the old v4 edition? AA Intelligence Index v4 · Bottom 20%#62
Q1 Bottom 14%#94 of 108 1402 Q2 84.2% Q3 18.9 About Arena Text More about Arena Text
Q1 Can it handle customer service by the rules? τ³-Banking · Bottom 42%#47 Q1 Can it handle customer service by the rules? τ³-Banking · Bottom 42%#47 Q2 Can it handle customer service by the rules? TAU2 · Bottom 22%#63 Q3 Can it finish long tasks that take people hours? METR Time Horizon · Bottom 20%#9 Q1 Bottom 42%#47 of 79 16.7% Q2 64.6% Q3 1h 15m Q4 33.1% Q5 43.9% About τ³-Banking More about τ³-Banking
Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Bottom 29%#88 Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Bottom 29%#88 Q2 Which AI gives the business answers people prefer most? Arena Business, Management & Finance · Bottom 11%#98 Q3 Can it read a contract hundreds of pages long and find exactly what's needed? CorpFin v2 · Bottom 34%#43 Q1 Bottom 29%#88 of 121 70.3% Q2 1381 Q3 61.2% Q4 35.0% Q5 46.9% Q6 44.5% By effort None 44.0% Thinking 70.3%
About Long Context Reasoning More about Long Context Reasoning
Q1 Can it carry out multi-step web research? Deep Research Bench · Bottom 36%#12 Q1 Can it carry out multi-step web research? Deep Research Bench · Bottom 36%#12 Q2 Does it avoid making things up when summarizing? Vectara Hallucination Rate · Bottom 45%#23 Q3 Does it avoid making things up when summarizing? Vectara Factual Consistency · Bottom 45%#23 Q4 How well can it predict what will happen? ForecastBench Brier Index · Bottom 34%#19
Q1 Bottom 36%#12 of 17 46.6% Reasoning effort: Budget 2K Q2 10.3% Q3 89.7% Q4 58.9 About Deep Research Bench More about Deep Research Bench
Q1 Which AI do people prefer most for coding answers? Arena Coding · Bottom 23%#85 Q1 Which AI do people prefer most for coding answers? Arena Coding · Bottom 23%#85 Q2 Can it find better answers to optimization problems with no single right answer? ALE-Bench · Bottom 20%#73 Q3 Can it operate a computer itself to finish development jobs on its own? Terminal-Bench 2.1 · Bottom 21%#65 Q1 Bottom 23%#85 of 108 1475 Q2 655 Q3 36.3% Q4 31.1% Q5 37.6 Q6 65.5% Q7 40.0% Q8 46.1% Q9 4.9% Q10 61.3% Q11 1446 About Arena Coding More about Arena Coding
Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Bottom 23%#85 Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Bottom 23%#85 Q2 Which AI do people prefer most in long conversations? Arena Multi-turn · Bottom 21%#87 Q3 Does it follow tricky rules like format and length without missing any? IFBench · Bottom 24%#63 Q1 Bottom 23%#85 of 108 1399 Q2 1421 Q3 54.7% Q4 1396 Q5 1416 Q6 8.14 Q7 72.4% Q8 1389 About Arena Writing, Literature & Language More about Arena Writing, Literature & Language
Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Bottom 12%#109 Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Bottom 12%#109 Q2 Can it solve research-level physics problems? CritPt · Bottom 11%#79 Q3 Which AI gives the math answers people prefer most? Arena Math · Bottom 14%#91 Q1 Bottom 12%#109 of 122 10.7% Q2 0.3% Q3 1404 Q4 77.7% Q5 71.1% Q6 74.3 Q7 74.3% Q8 1416 Q9 1412 Q10 99.1% Q11 84.4% Q12 4.1% Q13 0.0% Q14 40.0% Q15 5.9% Q16 45.5% Q17 3.1% Q18 77.1% Q19 29.0 By effort None 4.3% Thinking 10.7%
About Humanity's Last Exam More about Humanity's Last Exam
Q1 Which AI gives the photo and chart answers people prefer most? Arena Vision · Bottom 7%#63 Q1 Which AI gives the photo and chart answers people prefer most? Arena Vision · Bottom 7%#63 Q2 Can it predict how objects will move just from a picture? VPCT · Bottom 20%#9 Q3 Can it tell where a photo was taken from a single image? GeoBench · Bottom 5%#21 Q4 Can it understand a space from photos taken at different angles? MindCube ·
Q1 Bottom 7%#63 of 66 1191 Q2 34.0% Q3 37.0% Q4 44.8% About Arena Vision More about Arena Vision
Q1 Which AI do people prefer most when asking in Korean? Arena Korean · Bottom 13%#81 Q1 Which AI do people prefer most when asking in Korean? Arena Korean · Bottom 13%#81 Q2 Which AI do people prefer most when asking in Japanese? Arena Japanese · Bottom 14%#77 Q3 Which AI do people prefer most on tough questions? Arena Hard Prompts · Bottom 16%#92 Q1 Bottom 13%#81 of 91 1338 Q2 1343 Q3 1433 Q4 1434 Q5 1416 Q6 1406 Q7 35.0% Q8 17.9% Q9 56.1% Q10 28.6% Q11 20.4% Q12 66.3% Q13 28.6% Q14 90.6% Q15 90.0% Q16 91.3% Q17 97.8% Q18 81.5% Q19 28.4% Q20 79.2% Q21 89.3% Q22 74.6% Q23 6.0% Q24 24.6% About Arena Korean More about Arena Korean
Q1 Does it call out a nonsense question instead of playing along? BullshitBench v2 · Bottom 37%#92 Q1 Does it call out a nonsense question instead of playing along? BullshitBench v2 · Bottom 37%#92 Q2 Who survives longer in head-to-head snake games? SnakeBench · Top 10%#28
Q1 Bottom 37%#92 of 143 30.0% Reasoning effort: High Q2 32 By effort None 29.0% High 30.0%
About BullshitBench v2 More about BullshitBench v2