Q1 How strong is it across many tests combined? AA Intelligence Index · Bottom 14%#102 Q1 How strong is it across many tests combined? AA Intelligence Index · Bottom 14%#102 Q2 Which AI gives the answers people prefer most? Arena Text · Bottom 10%#89
Q1 Bottom 14%#102 of 117 12.8 Reasoning effort: Thinking Q2 1361 About AA Intelligence Index More about AA Intelligence Index
Q1 Can it handle customer service by the rules? τ³-Banking · Bottom 21%#64 Q1 Can it handle customer service by the rules? τ³-Banking · Bottom 21%#64 Q2 Can it handle customer service by the rules? TAU2 · Bottom 26%#60
Q1 Bottom 21%#64 of 79 10.3% Reasoning effort: Thinking Q2 67.8% About τ³-Banking More about τ³-Banking
Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Bottom 20%#94 Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Bottom 20%#94 Q2 Which AI gives the business answers people prefer most? Arena Business, Management & Finance · Bottom 8%#91
Q1 Bottom 20%#94 of 116 65.7% Reasoning effort: Thinking Q2 1350 About Long Context Reasoning More about Long Context Reasoning
Q1 Which AI gives answers people prefer most, without false claims? Arena Text Factuality · Bottom 8%#84 Q1 Which AI gives answers people prefer most, without false claims? Arena Text Factuality · Bottom 8%#84 Q2 Can it track down hard-to-find information on the web? BrowseComp · Bottom 8%#38 Q3 Can it use search and code to solve problems that stump experts? Humanity's Last Exam · Bottom 5%#43
Q1 Bottom 8%#84 of 90 1390 Q2 31.3% Q3 22.8% About Arena Text Factuality More about Arena Text Factuality
Q1 Can it operate a computer itself to finish hours-long development jobs? Terminal-Bench 4.0 · Bottom 27%#59 Q1 Can it operate a computer itself to finish hours-long development jobs? Terminal-Bench 4.0 · Bottom 27%#59 Q2 Which AI gives the coding answers people prefer most? Arena Coding · Bottom 11%#88 Q3 Can it find better answers to optimization problems with no single right answer? ALE-Bench · Bottom 3%#73 Q1 Bottom 27%#59 of 79 0.0% Reasoning effort: Thinking Q2 1407 Q3 214 Q4 38.6% Q5 28.8% Q6 37.7 Q7 36.2% Q8 38.0% Q9 1404 About Terminal-Bench 4.0 More about Terminal-Bench 4.0
Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Bottom 10%#89 Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Bottom 10%#89 Q2 Which AI do people prefer most in long conversations? Arena Multi-turn · Bottom 11%#88 Q3 Does it follow tricky rules like format and length without missing any? IFBench · Top 44%#35 Q1 Bottom 10%#89 of 97 1325 Q2 1351 Q3 71.5% Q4 1303 Q5 1342 Q6 1316 About Arena Writing, Literature & Language More about Arena Writing, Literature & Language
Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Bottom 30%#84 Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Bottom 30%#84 Q2 Can it solve research-level physics problems? CritPt · Bottom 31%#55 Q3 Which AI gives the math answers people prefer most? Arena Math · Bottom 10%#86 Q1 Bottom 30%#84 of 117 20.8% Reasoning effort: Thinking Q2 3.1% Q3 1373 Q4 80.0% Q5 1368 Q6 1399 About Humanity's Last Exam More about Humanity's Last Exam
Q1 Which AI do people prefer most on tough questions? Arena Hard Prompts · Bottom 10%#89 Q1 Which AI do people prefer most on tough questions? Arena Hard Prompts · Bottom 10%#89 Q2 Which AI do people prefer most on expert-level questions? Arena Expert · Bottom 11%#88 Q3 Which AI gives the medical and health answers people prefer most? Arena Medicine & Healthcare · Bottom 7%#91 Q1 Bottom 10%#89 of 97 1378 Q2 1393 Q3 1351 Q4 1366 Q5 87.0% Q6 64.1% Q7 63.2% Q8 65.1% Q9 93.0% Q10 83.0% Q11 90.9% Q12 59.3% Q13 41.7% Q14 85.0% Q15 66.0% Q16 96.7% Q17 20.5% Q18 17.3% Q19 39.0% Q20 8.5% Q21 77.0% Q22 66.0% Q23 57.0% Q24 80.0% Q25 53.0% Q26 82.3% Q27 50.3% Q28 60.0% Q29 75.0% Q30 50.4% Q31 54.4% Q32 82.5% Q33 61.1% Q34 73.6% About Arena Hard Prompts More about Arena Hard Prompts