Q1 How strong is it across many tests combined? AA Intelligence Index v5 · Top 3%#1 Q1 How strong is it across many tests combined? AA Intelligence Index v5 · Top 3%#1 Q2 Can it solve problems it never trained on? LiveBench 2.0 · Top 5%#1
Q1 #1 of 45 64.6 Reasoning effort: Max Q2 84.9 / 100 By effort Low 53.9 Medium 56.4 High 61.8 Extra High 62.2 Max 64.6
About AA Intelligence Index v5 More about AA Intelligence Index v5
Q1 Would a client accept its work on freelance jobs? Remote Labor Index · Top 8%#1 Q1 Would a client accept its work on freelance jobs? Remote Labor Index · Top 8%#1 Q2 Can it finish long tasks where early decisions must hold up to the end? LiveBench 2.0 Long Agentic · Top 5%#1 Q3 Can it complete multi-step tasks in a real web app using a browser? LiveBench 2.0 Browser · Top 32%#7
Q1 #1 of 14 41.3% Q2 82.9 / 100 Q3 86.9 / 100 About Remote Labor Index More about Remote Labor Index
Q1 Can it read tables and reshape them as needed? LiveBench 2.0 Data Analysis · Top 23%#5 Q1 Can it read tables and reshape them as needed? LiveBench 2.0 Data Analysis · Top 23%#5 Q2 Can it do professional work in banking, consulting and law? APEX-Agents · Top 6%#3 Q3 Can it research US corporate tax questions and answer with sources? Tax Agent Bench · Top 32%#20 Q1 #5 of 22 81.5 / 100 Reasoning effort: Extra High Q2 80.1% Q3 32.1% Q4 46.2% Q5 5.0% Q6 85.1 / 100 Q7 69.5% About LiveBench 2.0 Data Analysis More about LiveBench 2.0 Data Analysis
Q1 Does it know facts accurately without searching? SimpleQA Verified · Top 2%#1 Q1 Does it know facts accurately without searching? SimpleQA Verified · Top 2%#1 Q2 Can it research an open question and answer with a sourced report? LiveBench 2.0 Research · Top 5%#1 Q3 Can it carry out scientists' real research tasks in a terminal? Terminal-Bench-Science 0.1 · Top 6%#2
Q1 #1 of 65 94.0% Reasoning effort: Max Q2 94.5 / 100 Q3 55.7% About SimpleQA Verified More about SimpleQA Verified
Q1 Can it find better answers to optimization problems with no single right answer? ALE-Bench · Top 5%#4 Q1 Can it find better answers to optimization problems with no single right answer? ALE-Bench · Top 5%#4 Q2 Can it fix a bug without breaking anything else? LiveBench 2.0 Coding · Top 46%#10 Q3 Can it rebuild a program from scratch given only the executable and a spec? ProgramBench · Top 2%#1 Q1 #4 of 87 2510 Reasoning effort: High Q2 79.2 / 100 Q3 19.0% Q4 93.8% Q5 60.0% Q6 69.9% About ALE-Bench More about ALE-Bench
Q1 Can it solve research-level physics problems? CritPt · Top 3%#2 Q1 Can it solve research-level physics problems? CritPt · Top 3%#2 Q2 Can it solve competition-level math problems? OTIS Mock AIME 2024-2025 · Top 16%#12 Q3 Can it solve university- to research-level math problems? FrontierMath Tiers 1-3 v2 · Top 2%#1 Q1 #2 of 87 35.7% Reasoning effort: Extra High Q2 99.7% Q3 95.1% Q4 100.0% Q5 100.0% Q6 88.8 / 100 Q7 98.5% Q8 93.8% Q9 94.9% Q10 79.0% Q11 90.0% Q12 99.2% Q13 70.6 By effort Low 29.4% Medium 30.9% High 32.9% Extra High 35.7% Max 35.4%
About CritPt More about CritPt
Q Can it spot the wrong step from assembly photos? Furniture Assembly · Top 3%#1
Q Can it spot the wrong step from assembly photos? Furniture Assembly · Top 3%#1
#1 of 34 90.0% Reasoning effort: Max About Furniture Assembly More about Furniture Assembly
Q1 Can it learn the rules of a new board game on its own? EBR-bench · Top 4%#1 Q1 Can it learn the rules of a new board game on its own? EBR-bench · Top 4%#1 Q2 Kaggle Game Arena: Chess (text) Kaggle Game Arena: Chess · Top 21%#8 Q3 Kaggle Game Arena: Four in a Row Kaggle Game Arena: Four in a Row · Top 8%#3 Q4 Kaggle Game Arena Kaggle Game Arena · Top 8%#3
Q1 #1 of 28 90.5% Reasoning effort: Max Q2 1313 Q3 819 Q4 535 About EBR-bench More about EBR-bench
