Q1 Which AI gives the answers people prefer most? Arena Text · Top 39%#42 Q1 Which AI gives the answers people prefer most? Arena Text · Top 39%#42 Q2 Does it have broad knowledge across expert fields? MMLU-Pro · Top 4%#1 Q3 On the old v1, could it solve problems it had not seen in training? LiveBench Overall v1 · Bottom 33%#36 Q4 How high was its intelligence index on the old v4 edition? AA Intelligence Index v4 · Bottom 48%#41
Q1 Top 39%#42 of 108 1470 Q2 89.5% Q3 72.6 Q4 29.1 About Arena Text More about Arena Text
Q1 Can it run a small business alone for a year and make money? Vending-Bench 2 · Bottom 39%#36 Q1 Can it run a small business alone for a year and make money? Vending-Bench 2 · Bottom 39%#36 Q2 Can it handle customer service by the rules? TAU2 · Top 47%#37 Q3 Can it pick the right tools and use them properly? MCP-Atlas · Bottom 25%#29 Q1 Bottom 39%#36 of 57 $4,967 Q2 89.5% Q3 62.3% Q4 3.8% Q5 4h 53m Q6 66.3% Q7 84.0% Q8 79.6% About Vending-Bench 2 More about Vending-Bench 2
Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Bottom 49%#63 Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Bottom 49%#63 Q2 Which AI gives the business answers people prefer most? Arena Business, Management & Finance · Top 29%#31 Q3 Can its work match what professionals produce? GDPval · Top 20%#1 Q1 Bottom 49%#63 of 121 77.3% Q2 1472 Q3 45.5% Q4 65.1% Q5 49.2% Q6 1466 Q7 74.4 Q8 58.8% Q9 55.2% Q10 21.1% By effort None 70.7% Thinking 77.3%
About Long Context Reasoning More about Long Context Reasoning
Q1 Which AI gives answers people prefer most, without false claims? Arena Text Factuality · Top 44%#45 Q1 Which AI gives answers people prefer most, without false claims? Arena Text Factuality · Top 44%#45 Q2 Does it know facts accurately without searching? SimpleQA Verified · Bottom 45%#39 Q3 Can it carry out multi-step web research? Deep Research Bench · Top 18%#3 Q1 Top 44%#45 of 103 1459 Q2 45.7% Q3 54.8% Q4 67.8% Q5 43.2% Q6 24.0% Q7 1179 Q8 10.9% Q9 89.1% Q10 60.2 Q11 80.0% About Arena Text Factuality More about Arena Text Factuality
Q1 Which AI do people prefer most for coding answers? Arena Coding · Top 28%#30 Q1 Which AI do people prefer most for coding answers? Arena Coding · Top 28%#30 Q2 Can AI test and fix code itself until it completes the given task? LiveBench Agentic Coding v1 · Bottom 10%#48 Q3 Can it find better answers to optimization problems with no single right answer? ALE-Bench · Bottom 48%#48 Q1 Top 28%#30 of 108 1523 Q2 39.7 Q3 1025 Q4 52.0% Q5 47.0% Q6 47.8 Q7 1469 Q8 87.1% Q9 49.5% Q10 76.7% Q11 63.7% Q12 26.5% Q13 1508 Q14 79.7 Q15 20.6% Q16 63.1% Q17 1.77× About Arena Coding More about Arena Coding
Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Top 25%#27 Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Top 25%#27 Q2 Which AI do people prefer most in long conversations? Arena Multi-turn · Top 25%#26 Q3 Does it follow tricky rules like format and length without missing any? IFBench · Bottom 31%#57 Q1 Top 25%#27 of 108 1466 Q2 1484 Q3 58.0% Q4 1461 Q5 1474 Q6 81.3 Q7 85.3% Q8 1454 Q9 62.5 About Arena Writing, Literature & Language More about Arena Writing, Literature & Language
Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Bottom 47%#66 Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Bottom 47%#66 Q2 Which AI gives the math answers people prefer most? Arena Math · Top 50%#52 Q3 Can it solve graduate-level science problems? GPQA Diamond · Bottom 50%#52 Q1 Bottom 47%#66 of 122 30.1% Q2 1463 Q3 86.6% Q4 86.1% Q5 91.3 Q6 91.3% Q7 90.4 Q8 34.4% Q9 4.9% Q10 36.0% Q11 1487 Q12 1474 Q13 80.1 Q14 20.7% Q15 4.2% Q16 80.0% Q17 37.6% Q18 62.0% Q19 11.9% Q20 14.0% Q21 22.0% Q22 89.9% Q23 44.5 By effort None 13.2% Thinking 30.1%
About Humanity's Last Exam More about Humanity's Last Exam
Q1 Can it spot the wrong step from assembly photos? Furniture Assembly · Bottom 22%#31 Q1 Can it spot the wrong step from assembly photos? Furniture Assembly · Bottom 22%#31 Q2 Can it read charts and answer accurately? CharXiv · Bottom 7%#30 Q3 Can it predict how objects will move just from a picture? VPCT · Top 40%#4 Q1 Bottom 22%#31 of 38 28.3% Reasoning effort: Budget 64K Q2 68.7% Q3 40.0% Q4 75.0% Q5 70.6% About Furniture Assembly More about Furniture Assembly
Q1 Which AI do people prefer most when asking in Korean? Arena Korean · Top 22%#20 Q1 Which AI do people prefer most when asking in Korean? Arena Korean · Top 22%#20 Q2 Which AI do people prefer most when asking in Japanese? Arena Japanese · Top 45%#39 Q3 Can it use expert knowledge in many languages? MMMLU · Top 28%#5 Q1 Top 22%#20 of 91 1449 Q2 1439 Q3 90.8% Q4 1498 Q5 1503 Q6 1487 Q7 1483 Q8 82.0% Q9 50.6% Q10 43.5% Q11 14.3% Q12 62.1% Q13 39.2% Q14 30.6% Q15 73.2% Q16 39.2% Q17 91.0% Q18 77.5% Q19 78.9% Q20 76.0% Q21 94.0% Q22 86.5% Q23 94.1% Q24 83.4% Q25 302 Q26 91.0% Q27 86.0% Q28 93.3% Q29 61.0% Q30 22.0% Q31 82.0% Q32 89.6% Q33 95.0% Q34 79.0% Q35 80.0% Q36 93.0% Q37 65.0% Q38 42.3% Q39 80.6% Q40 88.3% Q41 77.3% Q42 76.3% Q43 39.0% Q44 9.0% About Arena Korean More about Arena Korean
Q1 Does it call out a nonsense question instead of playing along? BullshitBench v2 · Top 3%#4 Q1 Does it call out a nonsense question instead of playing along? BullshitBench v2 · Top 3%#4 Q2 Can it sort word-puzzle tiles into groups using meaning and wordplay? NYT Connections Extended · Bottom 34%#78 Q3 Who survives longer in head-to-head snake games? SnakeBench · Top 15%#45
Q1 Top 3%#4 of 143 90.0% Reasoning effort: High Q2 52.5% Q3 30 By effort None 79.0% High 90.0%
About BullshitBench v2 More about BullshitBench v2