Q1 How strong is it across many tests combined? AA Intelligence Index · Bottom 32%#81 Q1 How strong is it across many tests combined? AA Intelligence Index · Bottom 32%#81 Q2 Which AI gives the answers people prefer most? Arena Text · Bottom 23%#76 Q3 Does it have broad knowledge across expert fields? MMLU-Pro · Top 18%#5
Q1 #81 of 117 20.6 Reasoning effort: Thinking Q2 1426 Q3 87.3% By effort None 16.6 Thinking 20.6
About AA Intelligence Index More about AA Intelligence Index
Q1 Can it handle customer service by the rules? TAU2 · Bottom 30%#57 Q1 Can it handle customer service by the rules? TAU2 · Bottom 30%#57 Q2 Can it finish long tasks that take people hours? METR Time Horizon · Bottom 30%#8
Q1 #57 of 79 73.4% Reasoning effort: Thinking Q2 1h 40m About TAU2 More about TAU2
Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Bottom 25%#88 Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Bottom 25%#88 Q2 Which AI gives the business answers people prefer most? Arena Business, Management & Finance · Bottom 19%#80 Q3 Can it read long stories and read between the lines? Fiction.LiveBench · Top 50%#5 Q4 Can it find company filings and do financial analysis? Finance Agent v1 · Bottom 47%#9
Q1 #88 of 116 69.3% Reasoning effort: Thinking Q2 1409 Q3 61.1% Q4 43.6% About Long Context Reasoning More about Long Context Reasoning
Q1 Can it carry out multi-step web research? Deep Research Bench · Bottom 42%#11 Q1 Can it carry out multi-step web research? Deep Research Bench · Bottom 42%#11 Q2 Does it avoid making things up when summarizing? Vectara Hallucination Rate · Bottom 28%#27 Q3 Does it avoid making things up when summarizing? Vectara Factual Consistency · Bottom 28%#27 Q4 How well can it predict what will happen? ForecastBench Brier Index · Top 15%#4
Q1 #11 of 17 46.8% Reasoning effort: Budget 2K Q2 12.0% Q3 88.0% Q4 60.6 About Deep Research Bench More about Deep Research Bench
Q1 Which AI gives the coding answers people prefer most? Arena Coding · Bottom 41%#59 Q1 Which AI gives the coding answers people prefer most? Arena Coding · Bottom 41%#59 Q2 Can it operate a computer itself to crack the toughest development jobs? TerminalBench Hard · Bottom 37%#51 Q3 How strong is its overall coding? AA Coding Index · Bottom 23%#78 Q1 #59 of 97 1498 Q2 31.1% Q3 34.0 Q4 63.6% Q5 39.8% Q6 70.7% Q7 43.7% Q8 6.9% Q9 72.0% Q10 1467 Q11 1.33× About Arena Coding More about Arena Coding
Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Bottom 45%#55 Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Bottom 45%#55 Q2 Which AI do people prefer most in long conversations? Arena Multi-turn · Bottom 31%#68 Q3 Does it follow tricky rules like format and length without missing any? IFBench · Bottom 21%#65 Q1 #55 of 97 1431 Q2 1439 Q3 53.7% Q4 1432 Q5 1444 Q6 8.36 / 10 Q7 1422 About Arena Writing, Literature & Language More about Arena Writing, Literature & Language
Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Bottom 17%#99 Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Bottom 17%#99 Q2 Can it solve research-level physics problems? CritPt · Bottom 13%#69 Q3 Which AI gives the math answers people prefer most? Arena Math · Bottom 22%#75 Q1 #99 of 117 12.3% Reasoning effort: Thinking Q2 0.3% Q3 1421 Q4 79.6% Q5 64.4% Q6 73.3 Q7 73.3% Q8 1437 Q9 1426 Q10 98.2% Q11 85.0% Q12 4.5% Q13 4.2% Q14 35.7% Q15 8.6% Q16 58.8% Q17 5.6% Q18 81.6% Q19 37.4 By effort None 6.2% Thinking 12.3%
About Humanity's Last Exam More about Humanity's Last Exam
Q1 Which AI gives the photo and chart answers people prefer most? Arena Vision · Bottom 9%#54 Q1 Which AI gives the photo and chart answers people prefer most? Arena Vision · Bottom 9%#54 Q2 Can it predict how objects will move just from a picture? VPCT · Bottom 50%#6 Q3 Can it tell where a photo was taken from a single image? GeoBench · Bottom 29%#6
Q1 #54 of 58 1192 Q2 38.0% Q3 49.0% About Arena Vision More about Arena Vision
Q1 Which AI do people prefer most when asking in Korean? Arena Korean · Bottom 32%#55 Q1 Which AI do people prefer most when asking in Korean? Arena Korean · Bottom 32%#55 Q2 Which AI do people prefer most when asking in Japanese? Arena Japanese · Bottom 27%#56 Q3 Which AI do people prefer most on tough questions? Arena Hard Prompts · Bottom 30%#69 Q1 #55 of 79 1383 Q2 1387 Q3 1457 Q4 1448 Q5 1441 Q6 1433 Q7 38.0% Q8 88.0% Q9 73.6% Q10 79.6% Q11 67.6% Q12 87.0% Q13 84.2% Q14 94.2% Q15 78.9% Q16 86.0% Q17 84.0% Q18 66.7% Q19 35.0% Q20 13.0% Q21 79.0% Q22 8.8% Q23 96.0% Q24 72.0% Q25 73.0% Q26 95.0% Q27 71.0% Q28 78.0% Q29 88.8% Q30 73.0% Q31 6.0% Q32 25.1% About Arena Korean More about Arena Korean
