Q1 How strong is it across many tests combined? AA Intelligence Index · Bottom 25%#89 Q1 How strong is it across many tests combined? AA Intelligence Index · Bottom 25%#89 Q2 Which AI gives the answers people prefer most? Arena Text · Bottom 20%#79 Q3 Does it have broad knowledge across expert fields? MMLU-Pro · Bottom 25%#22
Q1 Bottom 25%#89 of 116 16.9 Reasoning effort: Thinking Q2 1414 Q3 80.0% By effort None 15.4 Thinking 16.9
About AA Intelligence Index More about AA Intelligence Index
Q1 Can it handle customer service by the rules? τ³-Banking · Bottom 14%#69 Q1 Can it handle customer service by the rules? τ³-Banking · Bottom 14%#69 Q2 Can it run a small business alone for a year and make money? Vending-Bench 2 · Bottom 11%#43 Q3 Can it handle customer service by the rules? TAU2 · Bottom 18%#66
Q1 Bottom 14%#69 of 79 9.3% Reasoning effort: Thinking Q2 $459 Q3 54.7% About τ³-Banking More about τ³-Banking
Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Bottom 44%#66 Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Bottom 44%#66 Q2 Which AI gives the business answers people prefer most? Arena Business, Management & Finance · Bottom 24%#75 Q3 Can it find company filings and do financial analysis? Finance Agent v2 · Bottom 9%#57 Q1 Bottom 44%#66 of 115 74.3% Reasoning effort: Thinking Q2 1420 Q3 31.0% Q4 60.6% Q5 28.3% Q6 10.6% Q7 0.8% Q8 32.7% Q9 1420 Q10 46.9% Q11 43.0% By effort None 49.7% Thinking 74.3%
About Long Context Reasoning More about Long Context Reasoning
Q1 Which AI gives answers people prefer most, without false claims? Arena Text Factuality · Bottom 15%#78 Q1 Which AI gives answers people prefer most, without false claims? Arena Text Factuality · Bottom 15%#78 Q2 Does it know facts accurately without searching? SimpleQA Verified · Bottom 8%#53 Q3 Can it carry out multi-step web research? Deep Research Bench · Bottom 30%#13 Q1 Bottom 15%#78 of 90 1418 Q2 13.2% Q3 45.5% Q4 12.8% Q5 9.8% Q6 90.2% Q7 59.0 About Arena Text Factuality More about Arena Text Factuality
Q1 Can it operate a computer itself to finish hours-long development jobs? Terminal-Bench 4.0 · Bottom 27%#58 Q1 Can it operate a computer itself to finish hours-long development jobs? Terminal-Bench 4.0 · Bottom 27%#58 Q2 Which AI gives the coding answers people prefer most? Arena Coding · Bottom 28%#71 Q3 Can it find better answers to optimization problems with no single right answer? ALE-Bench · Bottom 21%#60 Q1 Bottom 27%#58 of 78 0.0% Reasoning effort: Thinking Q2 1481 Q3 653 Q4 44.2% Q5 27.3% Q6 43.9 Q7 1329 Q8 61.5% Q9 42.2% Q10 45.4% Q11 1464 Q12 35.5% About Terminal-Bench 4.0 More about Terminal-Bench 4.0
Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Bottom 23%#76 Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Bottom 23%#76 Q2 Which AI do people prefer most in long conversations? Arena Multi-turn · Bottom 23%#76 Q3 Does it follow tricky rules like format and length without missing any? IFBench · Bottom 23%#64 Q1 Bottom 23%#76 of 97 1400 Q2 1425 Q3 54.3% Q4 1388 Q5 1415 Q6 1064 Q7 85.2% Q8 1389 About Arena Writing, Literature & Language More about Arena Writing, Literature & Language
Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Bottom 12%#104 Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Bottom 12%#104 Q2 Can it solve research-level physics problems? CritPt · Bottom 9%#72 Q3 Which AI gives the math answers people prefer most? Arena Math · Bottom 14%#82 Q1 Bottom 12%#104 of 116 10.4% Reasoning effort: Thinking Q2 0.0% Q3 1400 Q4 67.2% Q5 66.7% Q6 83.7 Q7 83.7% Q8 1430 Q9 1433 Q10 96.4% Q11 5.9% Q12 2.1% Q13 47.7% Q14 4.0% Q15 8.0% Q16 73.6% Q17 30.9 By effort None 4.2% Thinking 10.4%
About Humanity's Last Exam More about Humanity's Last Exam
Q Can it draw a floor plan from a few photos of a home? Blueprint-Bench 2 · Bottom 11%#27
Q Can it draw a floor plan from a few photos of a home? Blueprint-Bench 2 · Bottom 11%#27
Bottom 11%#27 of 29 0.0% About Blueprint-Bench 2 More about Blueprint-Bench 2
Q1 Which AI do people prefer most when asking in Korean? Arena Korean · Bottom 21%#64 Q1 Which AI do people prefer most when asking in Korean? Arena Korean · Bottom 21%#64 Q2 Which AI do people prefer most when asking in Japanese? Arena Japanese · Bottom 19%#62 Q3 Can it use expert knowledge in many languages? MMMLU · Bottom 12%#17 Q1 Bottom 21%#64 of 79 1366 Q2 1361 Q3 83.0% Q4 1441 Q5 1451 Q6 1425 Q7 1416 Q8 13.7% Q9 31.2% Q10 87.0% Q11 66.9% Q12 73.3% Q13 60.4% Q14 83.0% Q15 79.7% Q16 93.5% Q17 79.6% Q18 77.0% Q19 64.0% Q20 33.3% Q21 26.0% Q22 9.0% Q23 77.0% Q24 8.8% Q25 82.0% Q26 67.0% Q27 62.0% Q28 75.0% Q29 76.0% Q30 36.3% Q31 78.8% Q32 88.7% Q33 73.7% About Arena Korean More about Arena Korean