Q1 Which AI gives the answers people prefer most? Arena Text · Bottom 47%#59 Q1 Which AI gives the answers people prefer most? Arena Text · Bottom 47%#59 Q2 Does it have broad knowledge across expert fields? MMLU-Pro · Top 15%#4 Q3 How high was its intelligence index on the old v4 edition? AA Intelligence Index v4 · Bottom 27%#57
Q1 Bottom 47%#59 of 108 1455 Q2 87.5% Q3 20.7 About Arena Text More about Arena Text
Q1 Can it handle customer service by the rules? τ³-Banking · Top 42%#33 Q1 Can it handle customer service by the rules? τ³-Banking · Top 42%#33 Q2 Can it run a small business alone for a year and make money? Vending-Bench 2 · Bottom 29%#42 Q3 Can it handle customer service by the rules? TAU2 · Bottom 35%#53 Q1 Top 42%#33 of 79 24.5% Q2 $3,839 Q3 78.1% Q4 43.8% Q5 2.1% Q6 2h 2m Q7 62.9% Q8 72.0% Q9 72.4% About τ³-Banking More about τ³-Banking
Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Bottom 33%#83 Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Bottom 33%#83 Q2 Which AI gives the business answers people prefer most? Arena Business, Management & Finance · Top 46%#49 Q3 Can it do real work from many different jobs? JobBench · Bottom 10%#28 Q1 Bottom 33%#83 of 121 72.3% Q2 1460 Q3 20.7% Q4 42.5% Q5 62.0% Q6 44.1% Q7 1449 Q8 54.5% Q9 55.3% By effort None 54.0% Thinking 72.3%
About Long Context Reasoning More about Long Context Reasoning
Q1 Which AI gives answers people prefer most, without false claims? Arena Text Factuality · Bottom 43%#60 Q1 Which AI gives answers people prefer most, without false claims? Arena Text Factuality · Bottom 43%#60 Q2 Does it know facts accurately without searching? SimpleQA Verified · Bottom 18%#58 Q3 Can it carry out multi-step web research? Deep Research Bench · Top 30%#5 Q1 Bottom 43%#60 of 103 1449 Q2 30.7% Q3 52.6% Q4 43.9% Q5 33.6% Q6 16.0% Q7 1159 Q8 12.0% Q9 88.0% Q10 60.1 Q11 77.2% About Arena Text Factuality More about Arena Text Factuality
Q1 Can AI carry out long development tasks autonomously by operating a computer itself? Terminal-Bench 4.0 · Bottom 27%#63 Q1 Can AI carry out long development tasks autonomously by operating a computer itself? Terminal-Bench 4.0 · Bottom 27%#63 Q2 Which AI do people prefer most for coding answers? Arena Coding · Top 43%#46 Q3 Can it find better answers to optimization problems with no single right answer? ALE-Bench · Bottom 34%#61 Q1 Bottom 27%#63 of 84 0.0% Q2 1512 Q3 796 Q4 55.8% Q5 35.6% Q6 52.1 Q7 1385 Q8 71.4% Q9 45.7% Q10 71.3% Q11 47.7% Q12 14.7% Q13 1497 Q14 22.6% Q15 46.5% Q16 1.52× About Terminal-Bench 4.0 More about Terminal-Bench 4.0
Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Top 39%#42 Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Top 39%#42 Q2 Which AI do people prefer most in long conversations? Arena Multi-turn · Top 37%#39 Q3 Does it follow tricky rules like format and length without missing any? IFBench · Bottom 29%#59 Q1 Top 39%#42 of 108 1454 Q2 1477 Q3 57.3% Q4 1455 Q5 1462 Q6 84.5% Q7 1444 About Arena Writing, Literature & Language More about Arena Writing, Literature & Language
Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Bottom 23%#95 Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Bottom 23%#95 Q2 Can it solve research-level physics problems? CritPt · Bottom 20%#71 Q3 Which AI gives the math answers people prefer most? Arena Math · Bottom 25%#79 Q1 Bottom 23%#95 of 122 17.8% Q2 1.1% Q3 1426 Q4 83.4% Q5 77.8% Q6 88.0 Q7 88.0% Q8 23.9% Q9 2.4% Q10 19.0% Q11 1471 Q12 1450 Q13 97.7% Q14 15.2% Q15 4.2% Q16 63.7% Q17 13.6% Q18 54.3% Q19 6.0% Q20 10.0% Q21 17.0% Q22 83.2% Q23 38.8 By effort None 7.2% Thinking 17.8%
About Humanity's Last Exam More about Humanity's Last Exam
Q1 Can it predict how objects will move just from a picture? VPCT · Top 50%#5 Q1 Can it predict how objects will move just from a picture? VPCT · Top 50%#5 Q2 Can it tell where a photo was taken from a single image? GeoBench · Bottom 24%#17 Q3 Can it solve college-level problems with images and diagrams? MMMU-Pro · Bottom 17%#32
Q1 Top 50%#5 of 10 39.8% Reasoning effort: Budget 32K Q2 60.0% Q3 67.5% About VPCT More about VPCT
Q1 Which AI do people prefer most when asking in Korean? Arena Korean · Bottom 50%#46 Q1 Which AI do people prefer most when asking in Korean? Arena Korean · Bottom 50%#46 Q2 Which AI do people prefer most when asking in Japanese? Arena Japanese · Bottom 41%#53 Q3 Can it use expert knowledge in many languages? MMMLU · Bottom 50%#10 Q1 Bottom 50%#46 of 91 1419 Q2 1412 Q3 89.1% Q4 1483 Q5 1483 Q6 1473 Q7 1464 Q8 60.0% Q9 28.9% Q10 2.4% Q11 90.8% Q12 66.3% Q13 61.8% Q14 31.1% Q15 29.0% Q16 69.8% Q17 31.1% Q18 91.0% Q19 67.5% Q20 73.5% Q21 76.2% Q22 70.9% Q23 94.0% Q24 85.8% Q25 93.1% Q26 80.8% Q27 0 Q28 157 Q29 177 Q30 86.0% Q31 81.0% Q32 70.0% Q33 42.0% Q34 13.0% Q35 84.0% Q36 89.4% Q37 96.0% Q38 76.0% Q39 72.0% Q40 75.0% Q41 66.0% Q42 1.6% Q43 75.8% Q44 32.1% Q45 79.5% Q46 89.0% Q47 75.1% Q48 62.9% Q49 29.2% Q50 84.0% Q51 5.0% Q52 9.5% About Arena Korean More about Arena Korean
Q1 Does it call out a nonsense question instead of playing along? BullshitBench v2 · Top 9%#12 Q1 Does it call out a nonsense question instead of playing along? BullshitBench v2 · Top 9%#12 Q2 Can it sort word-puzzle tiles into groups using meaning and wordplay? NYT Connections Extended · Bottom 27%#86 Q3 Who survives longer in head-to-head snake games? SnakeBench · Top 8%#21 Q1 Top 9%#12 of 143 79.0% Reasoning effort: High Q2 37.3% Q3 33 Q4 325 Q5 566 Q6 150 By effort None 74.0% High 79.0%
About BullshitBench v2 More about BullshitBench v2