Q1 How strong is it across many tests combined? AA Intelligence Index · Bottom 32%#81 Q1 How strong is it across many tests combined? AA Intelligence Index · Bottom 32%#81 Q2 Which AI gives the answers people prefer most? Arena Text · Bottom 14%#85 Q3 Does it have broad knowledge across expert fields? MMLU-Pro · Top 47%#13
Q1 #81 of 117 20.6 Reasoning effort: Medium Q2 1390 Q3 83.7% By effort Minimal 9.90 Medium 20.6 High 16.8
About AA Intelligence Index More about AA Intelligence Index
Q1 Can it handle customer service by the rules? τ³-Banking · Bottom 40%#49 Q1 Can it handle customer service by the rules? τ³-Banking · Bottom 40%#49 Q2 Can it run a small business alone for a year and make money? Vending-Bench 2 · Bottom 3%#47 Q3 Can it handle customer service by the rules? TAU2 · Bottom 27%#59
Q1 #49 of 79 15.5% Reasoning effort: High Q2 −$31 Q3 71.1% About τ³-Banking More about τ³-Banking
Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Bottom 33%#79 Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Bottom 33%#79 Q2 Which AI gives the business answers people prefer most? Arena Business, Management & Finance · Bottom 14%#85 Q3 Can it read a contract hundreds of pages long and find exactly what's needed? CorpFin v2 · Bottom 21%#51 Q1 #79 of 116 72.3% Reasoning effort: High Q2 1381 Q3 60.2% Q4 43.0% Q5 69.4% Q6 51.9% By effort Minimal 39.3% Medium 72.3% High 72.3%
About Long Context Reasoning More about Long Context Reasoning
Q1 Which AI gives answers people prefer most, without false claims? Arena Text Factuality · Bottom 9%#83 Q1 Which AI gives answers people prefer most, without false claims? Arena Text Factuality · Bottom 9%#83 Q2 Does it know facts accurately without searching? SimpleQA Verified · Bottom 11%#51 Q3 Does it avoid making things up when summarizing? Vectara Hallucination Rate · Bottom 14%#32 Q1 #83 of 90 1394 Q2 21.6% Q3 12.9% Q4 87.1% Q5 60.4 About Arena Text Factuality More about Arena Text Factuality
Q1 Can it operate a computer itself to finish hours-long development jobs? Terminal-Bench 4.0 · Bottom 27%#59 Q1 Can it operate a computer itself to finish hours-long development jobs? Terminal-Bench 4.0 · Bottom 27%#59 Q2 Which AI gives the coding answers people prefer most? Arena Coding · Bottom 14%#85 Q3 Can it find better answers to optimization problems with no single right answer? ALE-Bench · Bottom 41%#45 Q1 #59 of 79 0.0% Reasoning effort: High Q2 1431 Q3 800 Q4 3.7% Q5 33.3% Q6 15.6 Q7 83.8% Q8 39.0% Q9 64.7% Q10 52.7% Q11 1416 Q12 34.8% Q13 51.0% Q14 1.38× Q15 23.0% About Terminal-Bench 4.0 More about Terminal-Bench 4.0
Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Bottom 13%#86 Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Bottom 13%#86 Q2 Which AI do people prefer most in long conversations? Arena Multi-turn · Bottom 12%#87 Q3 Does it follow tricky rules like format and length without missing any? IFBench · Top 30%#24 Q1 #86 of 97 1352 Q2 1373 Q3 75.4% Q4 1324 Q5 1373 Q6 8.31 / 10 Q7 80.6% Q8 1348 About Arena Writing, Literature & Language More about Arena Writing, Literature & Language
Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Bottom 31%#82 Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Bottom 31%#82 Q2 Can it solve research-level physics problems? CritPt · Bottom 9%#72 Q3 Which AI gives the math answers people prefer most? Arena Math · Bottom 18%#79 Q1 #82 of 117 21.5% Reasoning effort: High Q2 0.0% Q3 1405 Q4 82.8% Q5 86.7% Q6 90.7 Q7 90.7% Q8 46.7% Q9 12.2% Q10 9.0% Q11 1398 Q12 1407 Q13 97.8% Q14 27.2% Q15 6.3% Q16 54.3% Q17 4.4% Q18 8.2% Q19 30.0% Q20 10.0% Q21 80.5% Q22 34.2 By effort Minimal 5.1% Medium 15.9% High 21.5%
About Humanity's Last Exam More about Humanity's Last Exam
Q1 Which AI gives the photo and chart answers people prefer most? Arena Vision · Bottom 16%#50 Q1 Which AI gives the photo and chart answers people prefer most? Arena Vision · Bottom 16%#50 Q2 Can it read charts and answer accurately? CharXiv · Bottom 13%#28 Q3 Can it predict how objects will move just from a picture? VPCT · Top 30%#3 Q4 Can it solve college-level problems with images and diagrams? MMMU-Pro · Bottom 17%#32
Q1 #50 of 58 1202 Q2 75.5% Q3 40.2% Q4 67.5% About Arena Vision More about Arena Vision
Q1 Which AI do people prefer most when asking in Korean? Arena Korean · Bottom 12%#71 Q1 Which AI do people prefer most when asking in Korean? Arena Korean · Bottom 12%#71 Q2 Which AI do people prefer most when asking in Japanese? Arena Japanese · Bottom 20%#61 Q3 Which AI do people prefer most on tough questions? Arena Hard Prompts · Bottom 13%#86 Q1 #71 of 79 1322 Q2 1365 Q3 1404 Q4 1401 Q5 1407 Q6 1396 Q7 90.0% Q8 84.4% Q9 58.3% Q10 41.5% Q11 16.0% Q12 38.6% Q13 86.0% Q14 89.2% Q15 84.2% Q16 70.7% Q17 75.3% Q18 66.0% Q19 91.0% Q20 87.2% Q21 23.3% Q22 70.1% Q23 83.9% Q24 635 Q25 19 Q26 106 Q27 80.0% Q28 71.0% Q29 96.7% Q30 53.0% Q31 19.0% Q32 74.0% Q33 8.6% Q34 86.0% Q35 67.0% Q36 67.0% Q37 93.0% Q38 50.0% Q39 82.7% Q40 1.0% Q41 39.9% Q42 78.2% Q43 30.3% Q44 67.8% Q45 83.9% Q46 59.1% Q47 51.2% Q48 89.1% Q49 81.0% Q50 63.1% Q51 83.0% Q52 9.0% Q53 27.8% About Arena Korean More about Arena Korean
