Q1 Which AI gives the answers people prefer most? Arena Text · Top 38%#41 Q1 Which AI gives the answers people prefer most? Arena Text · Top 38%#41 Q2 On the old v1, could it solve problems it had not seen in training? LiveBench Overall v1 · Bottom 35%#35 Q3 How high was its intelligence index on the old v4 edition? AA Intelligence Index v4 · Bottom 49%#40
Q1 Top 38%#41 of 108 1472 Q2 73.0 Q3 30.1 About Arena Text More about Arena Text
Q1 Can it handle customer service by the rules? τ³-Banking · Top 32%#25 Q1 Can it handle customer service by the rules? τ³-Banking · Top 32%#25 Q2 Can it run a small business alone for a year and make money? Vending-Bench 2 · Top 39%#22 Q3 Can it handle customer service by the rules? TAU2 · Bottom 37%#51 Q1 Top 32%#25 of 79 34.4% Q2 $7,204 Q3 79.5% Q4 61.3% Q5 41.5% Q6 13.3% Q7 72.1% About τ³-Banking More about τ³-Banking
Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Top 30%#36 Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Top 30%#36 Q2 Which AI gives the business answers people prefer most? Arena Business, Management & Finance · Top 24%#25 Q3 Can it do professional work in banking, consulting and law? APEX-Agents · Bottom 24%#46 Q1 Top 30%#36 of 121 80.0% Q2 1481 Q3 43.0% Q4 36.6% Q5 51.0% Q6 65.3% Q7 25.8% Q8 38.5% Q9 1482 Q10 78.0 Q11 63.3% Q12 18.0% Q13 5.0% By effort Low (no thinking) 69.3% High (no thinking) 68.3% Max 80.0%
About Long Context Reasoning More about Long Context Reasoning
Q1 Which AI gives answers people prefer most, without false claims? Arena Text Factuality · Top 42%#43 Q1 Which AI gives answers people prefer most, without false claims? Arena Text Factuality · Top 42%#43 Q2 Does it know facts accurately without searching? SimpleQA Verified · Bottom 31%#49 Q3 Can it carry out multi-step web research? Deep Research Bench · Top 12%#2 Q1 Top 42%#43 of 103 1461 Q2 35.5% Q3 54.9% Q4 82.1% Q5 49.0% Q6 1220 Q7 10.6% Q8 89.4% Q9 59.6 Q10 91.1% About Arena Text Factuality More about Arena Text Factuality
Q1 Can AI carry out long development tasks autonomously by operating a computer itself? Terminal-Bench 4.0 · Bottom 48%#45 Q1 Can AI carry out long development tasks autonomously by operating a computer itself? Terminal-Bench 4.0 · Bottom 48%#45 Q2 Which AI do people prefer most for coding answers? Arena Coding · Top 21%#22 Q3 Can AI test and fix code itself until it completes the given task? LiveBench Agentic Coding v1 · Bottom 20%#43 Q1 Bottom 48%#45 of 84 3.0% Q2 1528 Q3 42.6 Q4 1327 Q5 58.1% Q6 71.2% Q7 53.0% Q8 63.0 Q9 1522 Q10 50.1% Q11 75.2% Q12 24.3% Q13 29.9% Q14 66.1% Q15 1514 Q16 79.3 Q17 0.5% Q18 51.5% Q19 53.4% Q20 48.8% About Terminal-Bench 4.0 More about Terminal-Bench 4.0
Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Top 38%#41 Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Top 38%#41 Q2 Which AI do people prefer most in long conversations? Arena Multi-turn · Top 30%#32 Q3 Does it follow tricky rules like format and length without missing any? IFBench · Bottom 28%#60 Q1 Top 38%#41 of 108 1455 Q2 1480 Q3 56.6% Q4 1449 Q5 1474 Q6 76.1 Q7 1197 Q8 1445 Q9 63.2 About Arena Writing, Literature & Language More about Arena Writing, Literature & Language
Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Top 50%#61 Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Top 50%#61 Q2 Can it solve research-level physics problems? CritPt · Bottom 29%#63 Q3 Which AI gives the math answers people prefer most? Arena Math · Bottom 50%#53 Q1 Top 50%#61 of 122 33.6% Q2 3.1% Q3 1461 Q4 87.5% Q5 85.8% Q6 87.0 Q7 45.0% Q8 1500 Q9 1482 Q10 84.8 Q11 32.4% Q12 8.3% Q13 86.5% Q14 60.4% Q15 23.0% Q16 16.0% Q17 89.9% Q18 46.5 By effort Low (no thinking) 11.2% High (no thinking) 13.3% Max 33.6%
About Humanity's Last Exam More about Humanity's Last Exam
Q1 Which AI gives the photo and chart answers people prefer most? Arena Vision · Top 37%#24 Q1 Which AI gives the photo and chart answers people prefer most? Arena Vision · Top 37%#24 Q2 Can it draw a floor plan from a few photos of a home? Blueprint-Bench 2 · Bottom 16%#33 Q3 Can it read charts and answer accurately? CharXiv · Bottom 26%#24 Q1 Top 37%#24 of 66 1283 Q2 6.7% Q3 77.4% Q4 73.0% Q5 74.5% About Arena Vision More about Arena Vision
Q1 Which AI do people prefer most when asking in Korean? Arena Korean · Top 32%#29 Q1 Which AI do people prefer most when asking in Korean? Arena Korean · Top 32%#29 Q2 Which AI do people prefer most when asking in Japanese? Arena Japanese · Top 44%#38 Q3 Can it use expert knowledge in many languages? MMMLU · Top 50%#9 Q1 Top 32%#29 of 91 1434 Q2 1440 Q3 89.3% Q4 1503 Q5 1506 Q6 1488 Q7 41.7% Q8 1484 Q9 65.2% Q10 23.6% Q11 56.2% Q12 32.6% Q13 76.4% Q14 32.6% Q15 92.0% Q16 75.3% Q17 75.0% Q18 75.7% Q19 89.0% Q20 87.5% Q21 98.4% Q22 82.6% Q23 98 Q24 133 Q25 203 Q26 -0.02 Q27 91.0% Q28 89.0% Q29 76.7% Q30 68.0% Q31 29.0% Q32 77.0% Q33 89.5% Q34 87.0% Q35 75.0% Q36 77.0% Q37 81.0% Q38 67.0% Q39 70.5% Q40 82.3% Q41 88.6% Q42 79.6% Q43 81.5% Q44 30.9% Q45 12.0% About Arena Korean More about Arena Korean
Q1 Does it call out a nonsense question instead of playing along? BullshitBench v2 · Top 3%#3 Q1 Does it call out a nonsense question instead of playing along? BullshitBench v2 · Top 3%#3 Q2 Can it sort word-puzzle tiles into groups using meaning and wordplay? NYT Connections Extended · Top 40%#46 Q3 Who survives longer in head-to-head snake games? SnakeBench · Top 25%#74 Q1 Top 3%#3 of 143 91.0% Reasoning effort: High Q2 80.9% Q3 26 Q4 348 Q5 296 Q6 689 Q7 215 By effort None 89.0% High 91.0%
About BullshitBench v2 More about BullshitBench v2