SemiAnalysis Unveils GDPx Work Benchmark

SemiAnalysis released GDPx on the 2nd local time, an AI agent benchmark measuring companies’ ability to perform professional work. It evaluates 206 tasks across 11 fields, including chip design, cybersecurity, biology, finance, law, software development, and AI self-improvement (systems improving their own capabilities). The company worked with 14 specialist firms to recreate real work environments. The leading models scored no higher than 12% overall.

Anthropic’s Claude Opus 5.5 ranked first at 12.0%, followed by Google’s Gemini 4 Carbon at 10.9%, OpenAI GPT-6 Astra at 9.6%, and Meta Muse Wonder Pool at 9.0%. These scores combine full and partial credit by task; they do not mean the models completed that share of 100 jobs. Each task allowed up to 8 hours, and scores were averaged across three runs.

SemiAnalysis said GDPx is intended to measure the ability to create economic value in real industries, rather than overall intelligence. The company selected high-value fields instead of weighting tasks by the overall distribution of jobs or their share of gross domestic product, reasoning that early revenue from AI adoption would concentrate in some professional industries where automation has a strong effect. It plans to use the results in its own analysis of OpenAI and Anthropic revenue forecasts for 2027–2028.

SemiAnalysis said GDPval, released by OpenAI last September, focused on providing data and instructions in advance and having models produce an output. Human professionals, it said, must find relevant information in large collections, compare conflicting material, and judge conditions that are not explicitly stated. GDPval was groundbreaking at release, the company said, but is now insufficient for measuring these abilities as AI agents have advanced. SemiAnalysis also said existing evaluations have included flawed answers or scoring criteria; when it applied its own quality checks to Terminal Bench 4.0, it found problems in about 50% of tasks.

GDPx is designed to check for errors in tasks and scoring criteria in environments containing real materials and programs. One finance task recreated a fictional environment based on an actual corporate acquisition, with about 4,000 anonymized files. The AI had to locate and analyze financial statements, contracts, and accounting records, then correct errors in existing documents. In a corporate profitability analysis task, no run achieved full success across 25 attempts; the best result passed 8 of 11 verification checks. In a legal task revising an investment agreement, four models made 20 attempts, and none met the completion criteria.

The top model varied by field. Opus 5.5 led in chip design, hardware design, and quantitative trading; Gemini 4 Carbon led in biology and software development; and GPT-6 Astra led in AI self-improvement and materials science. Artificial Analysis has also recently introduced complex-work evaluations, including AA Briefcase and FrontierCode 2.0, but GDPx focuses on expertise in specific industries and potential economic use. SemiAnalysis said GDPx scores therefore cannot be directly compared with existing overall indices to conclude that model performance is lower.

The evaluation used different agent execution environments for different models, including Claude Code and Codex, and safety restrictions applied to some cybersecurity and biology tasks. Some task content and scoring criteria were not disclosed, making it difficult for outsiders to independently reproduce the full results. SemiAnalysis said the findings showed differences between overall performance and abilities in specific fields, as well as between solving difficult problems and completing work from start to finish.

SemiAnalysis released GDPx on the 2nd local time, an AI agent benchmark measuring companies’ ability to perform professional work. It evaluates 206 tasks across 11 fields, including chip design, cybersecurity, biology, finance, law, software development, and AI self-improvement (systems improving their own capabilities). The company worked with 14 specialist firms to recreate real work environments. The leading models scored no higher than 12% overall.

Anthropic’s Claude Opus 5.5 ranked first at 12.0%, followed by Google’s Gemini 4 Carbon at 10.9%, OpenAI GPT-6 Astra at 9.6%, and Meta Muse Wonder Pool at 9.0%. These scores combine full and partial credit by task; they do not mean the models completed that share of 100 jobs. Each task allowed up to 8 hours, and scores were averaged across three runs.

SemiAnalysis said GDPx is intended to measure the ability to create economic value in real industries, rather than overall intelligence. The company selected high-value fields instead of weighting tasks by the overall distribution of jobs or their share of gross domestic product, reasoning that early revenue from AI adoption would concentrate in some professional industries where automation has a strong effect. It plans to use the results in its own analysis of OpenAI and Anthropic revenue forecasts for 2027–2028.

SemiAnalysis said GDPval, released by OpenAI last September, focused on providing data and instructions in advance and having models produce an output. Human professionals, it said, must find relevant information in large collections, compare conflicting material, and judge conditions that are not explicitly stated. GDPval was groundbreaking at release, the company said, but is now insufficient for measuring these abilities as AI agents have advanced. SemiAnalysis also said existing evaluations have included flawed answers or scoring criteria; when it applied its own quality checks to Terminal Bench 4.0, it found problems in about 50% of tasks.

GDPx is designed to check for errors in tasks and scoring criteria in environments containing real materials and programs. One finance task recreated a fictional environment based on an actual corporate acquisition, with about 4,000 anonymized files. The AI had to locate and analyze financial statements, contracts, and accounting records, then correct errors in existing documents. In a corporate profitability analysis task, no run achieved full success across 25 attempts; the best result passed 8 of 11 verification checks. In a legal task revising an investment agreement, four models made 20 attempts, and none met the completion criteria.

The top model varied by field. Opus 5.5 led in chip design, hardware design, and quantitative trading; Gemini 4 Carbon led in biology and software development; and GPT-6 Astra led in AI self-improvement and materials science. Artificial Analysis has also recently introduced complex-work evaluations, including AA Briefcase and FrontierCode 2.0, but GDPx focuses on expertise in specific industries and potential economic use. SemiAnalysis said GDPx scores therefore cannot be directly compared with existing overall indices to conclude that model performance is lower.

The evaluation used different agent execution environments for different models, including Claude Code and Codex, and safety restrictions applied to some cybersecurity and biology tasks. Some task content and scoring criteria were not disclosed, making it difficult for outsiders to independently reproduce the full results. SemiAnalysis said the findings showed differences between overall performance and abilities in specific fields, as well as between solving difficult problems and completing work from start to finish.