Q1 Which AI gives the answers people prefer most? Arena Text · Bottom 44%#62 Q1 Which AI gives the answers people prefer most? Arena Text · Bottom 44%#62 Q2 Does it have broad knowledge across expert fields? MMLU-Pro · Top 11%#3 Q3 How high was its intelligence index on the old v4 edition? AA Intelligence Index v4 · Bottom 31%#54
Q1 Bottom 44%#62 of 108 1450 Q2 88.0% Q3 22.8 About Arena Text More about Arena Text
Q1 Can it handle customer service by the rules? TAU2 · Bottom 28%#58 Q1 Can it handle customer service by the rules? TAU2 · Bottom 28%#58 Q2 Can it finish long tasks that take people hours? METR Time Horizon · Bottom 40%#7
Q1 Bottom 28%#58 of 79 71.4% Q2 1h 54m About TAU2 More about TAU2
Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Bottom 46%#67 Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Bottom 46%#67 Q2 Which AI gives the business answers people prefer most? Arena Business, Management & Finance · Bottom 42%#64 Q3 Can its work match what professionals produce? GDPval · Top 40%#2 Q1 Bottom 46%#67 of 121 76.0% Q2 1447 Q3 43.6% Q4 47.2% Q5 50.9% About Long Context Reasoning More about Long Context Reasoning
Q1 Which AI gives answers people prefer most, without false claims? Arena Text Factuality · Bottom 27%#77 Q1 Which AI gives answers people prefer most, without false claims? Arena Text Factuality · Bottom 27%#77 Q2 Can it carry out multi-step web research? Deep Research Bench · Bottom 50%#9 Q3 Which AI gives the search answers people prefer most? Arena Search · Bottom 14%#26 Q1 Bottom 27%#77 of 103 1435 Q2 48.3% Q3 1148 Q4 11.8% Q5 88.2% Q6 60.1 About Arena Text Factuality More about Arena Text Factuality
Q1 Which AI do people prefer most for coding answers? Arena Coding · Top 45%#48 Q1 Which AI do people prefer most for coding answers? Arena Coding · Top 45%#48 Q2 Can it find better answers to optimization problems with no single right answer? ALE-Bench · Bottom 24%#70 Q3 Can it operate a computer itself to crack the toughest development jobs? TerminalBench Hard · Bottom 45%#45 Q1 Top 45%#48 of 108 1511 Q2 675 Q3 34.3% Q4 36.5 Q5 65.4% Q6 40.9% Q7 73.3% Q8 45.9% Q9 1492 Q10 38.0% Q11 1.34× About Arena Coding More about Arena Coding
Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Bottom 50%#55 Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Bottom 50%#55 Q2 Which AI do people prefer most in long conversations? Arena Multi-turn · Top 46%#49 Q3 Does it follow tricky rules like format and length without missing any? IFBench · Bottom 25%#62 Q1 Bottom 50%#55 of 108 1446 Q2 1471 Q3 55.4% Q4 1446 Q5 1458 Q6 8.47 Q7 73.9% Q8 1435 About Arena Writing, Literature & Language More about Arena Writing, Literature & Language
Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Bottom 17%#103 Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Bottom 17%#103 Q2 Which AI gives the math answers people prefer most? Arena Math · Bottom 40%#64 Q3 Can it solve graduate-level science problems? GPQA Diamond · Bottom 29%#75 Q1 Bottom 17%#103 of 122 12.5% Q2 1444 Q3 80.9% Q4 68.9% Q5 80.3 Q6 80.3% Q7 12.6% Q8 2.4% Q9 1470 Q10 1451 Q11 7.2% Q12 4.2% Q13 60.0% Q14 7.2% Q15 21.0% Q16 80.0% Q17 37.1 About Humanity's Last Exam More about Humanity's Last Exam
Q Can it predict how objects will move just from a picture? VPCT · Bottom 30%#8
Q Can it predict how objects will move just from a picture? VPCT · Bottom 30%#8
Bottom 30%#8 of 10 35.0% About VPCT More about VPCT
Q1 Which AI do people prefer most when asking in Korean? Arena Korean · Top 47%#42 Q1 Which AI do people prefer most when asking in Korean? Arena Korean · Top 47%#42 Q2 Which AI do people prefer most when asking in Japanese? Arena Japanese · Bottom 41%#53 Q3 Which AI do people prefer most on tough questions? Arena Hard Prompts · Bottom 50%#56 Q1 Top 47%#42 of 91 1422 Q2 1412 Q3 1480 Q4 1482 Q5 1476 Q6 1463 Q7 42.0% Q8 7.9% Q9 54.8% Q10 33.1% Q11 33.2% Q12 65.0% Q13 33.1% Q14 94.3% Q15 94.0% Q16 95.0% Q17 90.0% Q18 75.9% Q19 80.0% Q20 71.8% Q21 93.0% Q22 85.0% Q23 94.1% Q24 82.1% Q25 83.0% Q26 83.0% Q27 70.0% Q28 46.0% Q29 16.0% Q30 83.0% Q31 88.9% Q32 92.0% Q33 74.0% Q34 72.0% Q35 99.0% Q36 66.0% Q37 29.9% Q38 79.9% Q39 89.8% Q40 75.0% Q41 31.0% Q42 8.0% Q43 25.3% About Arena Korean More about Arena Korean
Q Does it call out a nonsense question instead of playing along? BullshitBench v2 · Bottom 50%#72
Q Does it call out a nonsense question instead of playing along? BullshitBench v2 · Bottom 50%#72
Bottom 50%#72 of 143 43.0% Reasoning effort: None By effort None 43.0% High 42.0%
About BullshitBench v2 More about BullshitBench v2