Q1 Which AI gives the answers people prefer most? Arena Text · Top 50%#54 Q1 Which AI gives the answers people prefer most? Arena Text · Top 50%#54 Q2 How high was its intelligence index on the old v4 edition? AA Intelligence Index v4 · Bottom 47%#42
Q1 Top 50%#54 of 108 1457 Q2 27.9 About Arena Text More about Arena Text
Q1 Can it run a small business alone for a year and make money? Vending-Bench 2 · Bottom 36%#38 Q1 Can it run a small business alone for a year and make money? Vending-Bench 2 · Bottom 36%#38 Q2 Can it handle customer service by the rules? TAU2 · Top 8%#6 Q3 Can it pick the right tools and use them properly? MCP-Atlas · Bottom 33%#26 Q1 Bottom 36%#38 of 57 $4,432 Q2 98.2% Q3 67.8% Q4 82.5% Q5 73.7% About Vending-Bench 2 More about Vending-Bench 2
Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Bottom 43%#70 Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Bottom 43%#70 Q2 Which AI gives the business answers people prefer most? Arena Business, Management & Finance · Bottom 47%#59 Q3 Can it read a contract hundreds of pages long and find exactly what's needed? CorpFin v2 · Bottom 42%#38 Q1 Bottom 43%#70 of 121 75.7% Q2 1452 Q3 62.9% Q4 53.2% Q5 18.7% By effort None 43.7% Thinking 75.7%
About Long Context Reasoning More about Long Context Reasoning
Q1 Which AI gives answers people prefer most, without false claims? Arena Text Factuality · Bottom 49%#54 Q1 Which AI gives answers people prefer most, without false claims? Arena Text Factuality · Bottom 49%#54 Q2 Can it track down hard-to-find information on the web? BrowseComp · Bottom 25%#31 Q3 Can it use search and code to solve problems that stump experts? Humanity's Last Exam · Bottom 39%#28 Q1 Bottom 49%#54 of 103 1452 Q2 62.0% Q3 50.4% Q4 10.1% Q5 89.9% Q6 60.2 About Arena Text Factuality More about Arena Text Factuality
Q1 Which AI do people prefer most for coding answers? Arena Coding · Bottom 38%#68 Q1 Which AI do people prefer most for coding answers? Arena Coding · Bottom 38%#68 Q2 Can it find better answers to optimization problems with no single right answer? ALE-Bench · Bottom 30%#64 Q3 Can it fix real issues in real software projects? SWE-Bench Pro · Bottom 36%#35 Q1 Bottom 38%#68 of 108 1498 Q2 766 Q3 55.1% Q4 43.2% Q5 44.2 Q6 1435 Q7 46.2% Q8 72.1% Q9 48.2% Q10 1488 Q11 23.4% Q12 52.4% About Arena Coding More about Arena Coding
Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Top 49%#52 Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Top 49%#52 Q2 Which AI do people prefer most in long conversations? Arena Multi-turn · Top 45%#48 Q3 Does it follow tricky rules like format and length without missing any? IFBench · Top 41%#33 Q1 Top 49%#52 of 108 1447 Q2 1471 Q3 72.3% Q4 1447 Q5 1447 Q6 1438 About Arena Writing, Literature & Language More about Arena Writing, Literature & Language
Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Bottom 43%#71 Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Bottom 43%#71 Q2 Which AI gives the math answers people prefer most? Arena Math · Bottom 37%#67 Q3 Can it solve graduate-level science problems? GPQA Diamond · Bottom 32%#72 Q1 Bottom 43%#71 of 122 29.3% Q2 1443 Q3 82.0% Q4 80.0% Q5 82.5% Q6 1479 Q7 1455 Q8 16.4% Q9 2.1% Q10 44.7% Q11 4.9% Q12 53.2% By effort None 7.6% Thinking 29.3%
About Humanity's Last Exam More about Humanity's Last Exam
Q1 Which AI do people prefer most when asking in Korean? Arena Korean · Top 44%#40 Q1 Which AI do people prefer most when asking in Korean? Arena Korean · Top 44%#40 Q2 Which AI do people prefer most when asking in Japanese? Arena Japanese · Bottom 43%#52 Q3 Which AI do people prefer most on tough questions? Arena Hard Prompts · Bottom 49%#57 Q1 Top 44%#40 of 91 1424 Q2 1415 Q3 1478 Q4 1481 Q5 1479 Q6 1468 Q7 43.2% Q8 96.7% Q9 99.2% Q10 93.1% Q11 89.0% Q12 97.5% Q13 86.4% Q14 94.2% Q15 74.3% Q16 77.0% Q17 71.7% Q18 86.6% Q19 89.7% Q20 95.8% Q21 84.2% Q22 91.7% Q23 73.0% Q24 88.5% Q25 43.8% Q26 26.8% Q27 74.4% Q28 88.0% Q29 89.0% Q30 82.0% Q31 72.7% Q32 87.9% Q33 61.6% Q34 10.9% Q35 68.6% Q36 78.8% Q37 84.2% Q38 77.1% Q39 91.0% Q40 35.1% About Arena Korean More about Arena Korean
Q1 Does it call out a nonsense question instead of playing along? BullshitBench v2 · Bottom 35%#94 Q1 Does it call out a nonsense question instead of playing along? BullshitBench v2 · Bottom 35%#94 Q2 Can it sort word-puzzle tiles into groups using meaning and wordplay? NYT Connections Extended · Bottom 50%#59 Q3 Who survives longer in head-to-head snake games? SnakeBench · Top 15%#42
Q1 Bottom 35%#94 of 143 28.0% Reasoning effort: High Q2 74.8% Q3 31 By effort None 20.0% High 28.0%
About BullshitBench v2 More about BullshitBench v2