Q1 How strong is it across many tests combined? AA Intelligence Index · Bottom 42%#70 Q1 How strong is it across many tests combined? AA Intelligence Index · Bottom 42%#70 Q2 Which AI gives the answers people prefer most? Arena Text · Bottom 30%#69 Q3 Does it have broad knowledge across expert fields? MMLU-Pro · Top 22%#6
Q1 #70 of 117 23.0 Reasoning effort: High Q2 1435 Q3 87.1% By effort Minimal 11.4 Low 20.8 Medium 22.9 High 23.0
About AA Intelligence Index More about AA Intelligence Index
Q1 Can it handle customer service by the rules? τ³-Banking · Top 47%#37 Q1 Can it handle customer service by the rules? τ³-Banking · Top 47%#37 Q2 Can it handle customer service by the rules? TAU2 · Bottom 49%#42 Q3 Would a client accept its work on freelance jobs? Remote Labor Index · Bottom 10%#11 Q4 Can it finish long tasks that take people hours? METR Time Horizon · Top 50%#5
Q1 #37 of 79 22.1% Reasoning effort: High Q2 86.5% Q3 1.7% Q4 3h 23m About τ³-Banking More about τ³-Banking
Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Top 44%#51 Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Top 44%#51 Q2 Which AI gives the business answers people prefer most? Arena Business, Management & Finance · Bottom 21%#78 Q3 Can its work match what professionals produce? GDPval · Bottom 40%#4 Q1 #51 of 116 78.2% Reasoning effort: High Q2 1413 Q3 34.8% Q4 61.1% Q5 49.6% Q6 97.2% Q7 52.2% Q8 46.9% By effort Medium 76.0% High 78.2%
About Long Context Reasoning More about Long Context Reasoning
Q1 Which AI gives answers people prefer most, without false claims? Arena Text Factuality · Bottom 18%#75 Q1 Which AI gives answers people prefer most, without false claims? Arena Text Factuality · Bottom 18%#75 Q2 Does it know facts accurately without searching? SimpleQA Verified · Top 45%#25 Q3 Can it carry out multi-step web research? Deep Research Bench · Top 48%#8 Q1 #75 of 90 1421 Q2 50.1% Q3 49.6% Q4 59.4% Q5 61.3 About Arena Text Factuality More about Arena Text Factuality
Q1 Which AI gives the coding answers people prefer most? Arena Coding · Bottom 23%#76 Q1 Which AI gives the coding answers people prefer most? Arena Coding · Bottom 23%#76 Q2 Can it find better answers to optimization problems with no single right answer? ALE-Bench · Top 38%#28 Q3 Can it operate a computer itself to finish development jobs on its own? Terminal-Bench 2.1 · Bottom 19%#67 Q1 #76 of 97 1470 Q2 1162 Q3 35.2% Q4 37.9% Q5 37.8 Q6 84.6% Q7 37.8% Q8 73.6% Q9 60.7% Q10 6.9% Q11 88.0% Q12 1454 Q13 49.6% Q14 55.0% Q15 1.67× Q16 24.0% About Arena Coding More about Arena Coding
Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Bottom 21%#78 Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Bottom 21%#78 Q2 Which AI do people prefer most in long conversations? Arena Multi-turn · Bottom 21%#78 Q3 Does it follow tricky rules like format and length without missing any? IFBench · Top 38%#30 Q1 #78 of 97 1396 Q2 1420 Q3 73.1% Q4 1373 Q5 1410 Q6 8.60 / 10 Q7 83.7% Q8 1398 About Arena Writing, Literature & Language More about Arena Writing, Literature & Language
Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Bottom 43%#68 Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Bottom 43%#68 Q2 Can it solve research-level physics problems? CritPt · Top 49%#38 Q3 Which AI gives the math answers people prefer most? Arena Math · Bottom 31%#66 Q1 #68 of 117 28.5% Reasoning effort: High Q2 12.6% Q3 1434 Q4 85.4% Q5 91.4% Q6 94.3 Q7 94.3% Q8 55.4% Q9 22.0% Q10 18.0% Q11 1444 Q12 1442 Q13 99.4% Q14 98.1% Q15 32.4% Q16 12.5% Q17 65.7% Q18 9.9% Q19 56.7% Q20 10.5% Q21 37.0% Q22 23.0% Q23 90.7% Q24 40.0 By effort Minimal 6.0% Low 19.6% Medium 25.4% High 28.5%
About Humanity's Last Exam More about Humanity's Last Exam
Q1 Which AI gives the photo and chart answers people prefer most? Arena Vision · Bottom 18%#49 Q1 Which AI gives the photo and chart answers people prefer most? Arena Vision · Bottom 18%#49 Q2 Can it read charts and answer accurately? CharXiv · Bottom 49%#17 Q3 Can it predict how objects will move just from a picture? VPCT · Top 20%#2 Q4 Can it tell where a photo was taken from a single image? GeoBench · Top 29%#2
Q1 #49 of 58 1208 Q2 81.1% Q3 66.0% Q4 81.0% About Arena Vision More about Arena Vision
Q1 Which AI do people prefer most when asking in Korean? Arena Korean · Bottom 36%#52 Q1 Which AI do people prefer most when asking in Korean? Arena Korean · Bottom 36%#52 Q2 Which AI do people prefer most when asking in Japanese? Arena Japanese · Top 47%#35 Q3 Which AI do people prefer most on tough questions? Arena Hard Prompts · Bottom 27%#72 Q1 #52 of 79 1390 Q2 1428 Q3 1448 Q4 1460 Q5 1459 Q6 46.2% Q7 1459 Q8 39.4% Q9 32.8% Q10 12.7% Q11 91.7% Q12 90.0% Q13 69.6% Q14 44.1% Q15 55.8% Q16 56.5% Q17 89.0% Q18 88.8% Q19 91.5% Q20 89.0% Q21 88.3% Q22 89.2% Q23 75.5% Q24 78.0% Q25 73.1% Q26 93.0% Q27 92.9% Q28 38.1% Q29 95.9% Q30 85.8% Q31 85.8% Q32 83.0% Q33 81.0% Q34 90.0% Q35 68.0% Q36 26.0% Q37 86.0% Q38 8.6% Q39 80.0% Q40 80.0% Q41 77.0% Q42 96.0% Q43 54.0% Q44 90.7% Q45 1.0% Q46 78.8% Q47 52.6% Q48 79.7% Q49 88.2% Q50 75.9% Q51 65.8% Q52 62.4% Q53 92.0% Q54 85.4% Q55 68.5% Q56 84.2% Q57 7.0% Q58 26.2% About Arena Korean More about Arena Korean
