Q1 How strong is it across many tests combined? AA Intelligence Index · Bottom 15%#101 Q1 How strong is it across many tests combined? AA Intelligence Index · Bottom 15%#101 Q2 Which AI gives the answers people prefer most? Arena Text · Bottom 8%#91 Q3 Does it have broad knowledge across expert fields? MMLU-Pro · Bottom 18%#24
Q1 #101 of 117 13.0 Reasoning effort: High Q2 1338 Q3 78.0% By effort Minimal 7.10 Medium 12.5 High 13.0
About AA Intelligence Index More about AA Intelligence Index
Q Can it handle customer service by the rules? TAU2 · Bottom 9%#73
Q Can it handle customer service by the rules? TAU2 · Bottom 9%#73
#73 of 79 36.5% Reasoning effort: High By effort Minimal 25.7% Medium 30.4% High 36.5%
About TAU2 More about TAU2
Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Bottom 11%#105 Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Bottom 11%#105 Q2 Which AI gives the business answers people prefer most? Arena Business, Management & Finance · Bottom 7%#92 Q3 Can it read hospital records and assign the right diagnosis codes? MedCode · Bottom 7%#68 Q4 Can it read long stories and read between the lines? Fiction.LiveBench · Bottom 30%#8
Q1 #105 of 116 45.0% Reasoning effort: High Q2 1339 Q3 30.4% Q4 44.4% By effort Minimal 20.0% Medium 43.7% High 45.0%
About Long Context Reasoning More about Long Context Reasoning
Q1 Does it know facts accurately without searching? SimpleQA Verified · Bottom 6%#54 Q1 Does it know facts accurately without searching? SimpleQA Verified · Bottom 6%#54 Q2 Does it avoid making things up when summarizing? Vectara Hallucination Rate · Bottom 42%#22 Q3 Does it avoid making things up when summarizing? Vectara Factual Consistency · Bottom 42%#22 Q4 How well can it predict what will happen? ForecastBench Brier Index · Bottom 19%#23
Q1 #54 of 56 11.7% Reasoning effort: High Q2 10.5% Q3 89.5% Q4 57.2 About SimpleQA Verified More about SimpleQA Verified
Q1 Which AI gives the coding answers people prefer most? Arena Coding · Bottom 7%#92 Q1 Which AI gives the coding answers people prefer most? Arena Coding · Bottom 7%#92 Q2 Can it find better answers to optimization problems with no single right answer? ALE-Bench · Bottom 30%#53 Q3 Can it operate a computer itself to crack the toughest development jobs? TerminalBench Hard · Bottom 19%#65 Q1 #92 of 97 1384 Q2 719 Q3 17.4% Q4 22.9 Q5 78.9% Q6 33.8% Q7 38.1% Q8 1372 Q9 21.8% About Arena Coding More about Arena Coding
Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Bottom 4%#95 Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Bottom 4%#95 Q2 Which AI do people prefer most in long conversations? Arena Multi-turn · Bottom 7%#92 Q3 Does it follow tricky rules like format and length without missing any? IFBench · Bottom 47%#44 Q1 #95 of 97 1289 Q2 1324 Q3 67.6% Q4 1247 Q5 1327 Q6 72.9% Q7 1277 About Arena Writing, Literature & Language More about Arena Writing, Literature & Language
Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Bottom 9%#108 Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Bottom 9%#108 Q2 Which AI gives the math answers people prefer most? Arena Math · Bottom 7%#89 Q3 Can it solve graduate-level science problems? GPQA Diamond · Bottom 10%#94 Q1 #108 of 117 9.5% Reasoning effort: High Q2 1346 Q3 67.6% Q4 81.1% Q5 83.7 Q6 83.7% Q7 20.0% Q8 2.4% Q9 12.0% Q10 1348 Q11 1355 Q12 95.2% Q13 8.3% Q14 2.1% Q15 20.7% Q16 2.6% Q17 27.0% Q18 9.0% Q19 62.7% Q20 7.93 By effort Minimal 4.0% Medium 8.7% High 9.5%
About Humanity's Last Exam More about Humanity's Last Exam
Q1 Which AI gives the photo and chart answers people prefer most? Arena Vision · Bottom 6%#56 Q1 Which AI gives the photo and chart answers people prefer most? Arena Vision · Bottom 6%#56 Q2 Can it predict how objects will move just from a picture? VPCT · Bottom 40%#7
Q1 #56 of 58 1159 Q2 37.2% About Arena Vision More about Arena Vision
Q1 Which AI do people prefer most when asking in Korean? Arena Korean · Bottom 11%#72 Q1 Which AI do people prefer most when asking in Korean? Arena Korean · Bottom 11%#72 Q2 Which AI do people prefer most on tough questions? Arena Hard Prompts · Bottom 7%#92 Q3 Which AI do people prefer most on expert-level questions? Arena Expert · Bottom 8%#91 Q1 #72 of 79 1278 Q2 1356 Q3 1354 Q4 1366 Q5 1326 Q6 80.8% Q7 73.8% Q8 80.0% Q9 74.2% Q10 81.7% Q11 65.9% Q12 74.3% Q13 57.5% Q14 92.0% Q15 86.8% Q16 94.0% Q17 79.2% Q18 76.0% Q19 63.0% Q20 80.0% Q21 27.0% Q22 6.0% Q23 77.0% Q24 8.5% Q25 78.0% Q26 53.0% Q27 42.0% Q28 95.0% Q29 54.0% Q30 6.3% Q31 71.7% Q32 81.4% Q33 68.2% Q34 85.4% Q35 3.0% Q36 21.6% About Arena Korean More about Arena Korean
