Q1 How strong is it across many tests combined? AA Intelligence Index v5 · Top 31%#12 Q1 How strong is it across many tests combined? AA Intelligence Index v5 · Top 31%#12 Q2 Which AI gives the answers people prefer most? Arena Text · Top 3%#3 Q3 How strong was it across many tests combined, on the old v4 index? AA Intelligence Index v4 · Top 8%#6
Q1 Top 31%#12 of 39 43.6 Reasoning effort: Max Q2 1513 Q3 52.4 By effort High 40.5 Max 43.6
About AA Intelligence Index v5 More about AA Intelligence Index v5
Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Top 27%#32 Q1 Can it pull scattered facts from several long documents into one answer? Long Context Reasoning · Top 27%#32 Q2 Which AI gives the business answers people prefer most? Arena Business, Management & Finance · Top 3%#3 Q3 Can it do real work from many different jobs? JobBench · Top 7%#2 Q1 Top 27%#32 of 120 80.7% Q2 1510 Q3 75.3% Q4 59.1% Q5 43.0% Q6 53.4% Q7 23.8% Q8 49.9% Q9 67.6% Q10 21.7% About Long Context Reasoning More about Long Context Reasoning
Q Can it carry out scientists' real research tasks in a terminal? Terminal-Bench-Science 0.1 · Top 20%#7
Q Can it carry out scientists' real research tasks in a terminal? Terminal-Bench-Science 0.1 · Top 20%#7
Top 20%#7 of 36 37.1% Reasoning effort: Max About Terminal-Bench-Science 0.1 More about Terminal-Bench-Science 0.1
Q1 Can AI carry out long development tasks autonomously by operating a computer itself? Terminal-Bench 4.0 · Top 11%#9 Q1 Can AI carry out long development tasks autonomously by operating a computer itself? Terminal-Bench 4.0 · Top 11%#9 Q2 Which AI do people prefer most for coding answers? Arena Coding · Top 5%#5 Q3 Which AI builds the websites and apps people prefer most? Arena WebDev · Top 16%#11 Q1 Top 11%#9 of 83 52.0% Q2 1548 Q3 1650 Q4 60.3% Q5 1542 Q6 0.5% Q7 86.0% About Terminal-Bench 4.0 More about Terminal-Bench 4.0
Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Top 5%#5 Q1 Which AI gives the writing and literature answers people prefer most? Arena Writing, Literature & Language · Top 5%#5 Q2 Which AI do people prefer most in long conversations? Arena Multi-turn · Top 3%#3 Q3 Which AI writes the creative pieces people prefer most? Arena Creative Writing · Top 5%#5 Q1 Top 5%#5 of 101 1499 Q2 1525 Q3 1492 Q4 1501 Q5 1325 Q6 89.3% Q7 1491 About Arena Writing, Literature & Language More about Arena Writing, Literature & Language
Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Top 9%#10 Q1 Can it solve problems that stump experts across fields? Humanity's Last Exam · Top 9%#10 Q2 Which AI gives the math answers people prefer most? Arena Math · Top 17%#17 Q3 Which AI gives the science answers people prefer most? Arena Life, Physical & Social Science · Top 3%#3 Q4 Which AI gives the math answers people prefer most? Arena Mathematical · Top 12%#12
Q1 Top 9%#10 of 121 50.1% Q2 1494 Q3 1526 Q4 1509 About Humanity's Last Exam More about Humanity's Last Exam
Q1 Which AI do people prefer most when asking in Japanese? Arena Japanese · Top 3%#2 Q1 Which AI do people prefer most when asking in Japanese? Arena Japanese · Top 3%#2 Q2 Which AI do people prefer most on tough questions? Arena Hard Prompts · Top 3%#3 Q3 Which AI do people prefer most on expert-level questions? Arena Expert · Top 11%#11 Q1 Top 3%#2 of 88 1548 Q2 1533 Q3 1529 Q4 1521 Q5 1520 About Arena Japanese More about Arena Japanese