Chinese Models Trail in Revised AI Rankings

Artificial Analysis says Chinese open-weight AI models now trail the overall leader by 12 points in its revised ranking, compared with a 3-point gap under the previous system. The firm recently changed its tests to better reflect how people use AI, keeping many of them secret so companies cannot train specifically for known questions. Under the new system, publicly released-weight models scored 32 points, while Anthropic’s Claude Opus 5.5, the overall leader at the time, scored 58. Moonshot’s Kimi K3.1 Pro was also included in the 12-point gap reported by the firm.

The ranking has renewed questions about whether lower-cost, freely available Chinese AI models pose as much competitive pressure as investors and businesses have feared. Chinese companies have released models that businesses can download and adapt, often at lower cost than American alternatives. That has raised concerns that OpenAI and Anthropic could struggle to charge premium prices for proprietary models. The article says the new results may strengthen the investment case for Anthropic’s planned IPO and a possible future OpenAI listing, while noting this is an implication of the rankings rather than a confirmed outcome.

Artificial Analysis CEO Sridhar Ramaswamy said the gap between open models and the leader expanded from 3 points to 12 when benchmarks became private, and asked whether earlier tests had been leaking. When firms know the tests in advance, they can train models to score well on those tasks without necessarily matching that performance in real-world use; Microsoft CEO Satya Nadella and other experts call this “benchmark hacking.” The revised results indicate a larger US lead in complex scientific tasks: even Anthropic’s smaller Claude Haiku 5.5 outperformed every model with publicly released weights. Trillium Labs cofounder Nathan Lambert said the earlier index had likely rated open-weight models too favorably.

Software developer OpenCode cofounder Dax Raad examined traffic through OpenRouter, a service that gives developers access to models from different providers. He estimated that almost all DeepSeek traffic from the previous day on OpenRouter could be handled by one rack of Nvidia computers costing roughly $6 million to $7 million. OpenRouter captures only part of the market, so those figures do not measure all demand for open-weight models. Raad argued that much of the sector’s infrastructure spending comes from major providers such as OpenAI and Anthropic, rather than smaller developers using open models. ChatGPT has more than 1 billion monthly users, whose activity may exceed what AI gateways such as OpenRouter capture.

The rankings and usage estimate leave open two questions: whether open-weight models match leading American models and whether demand for them is as large as the attention suggests. Lambert said they remain competitive in areas such as software development and can give businesses more control and potentially lower costs. The evidence challenges the claim that Chinese AI companies are rapidly catching US leaders and threatening their business models, a distinction the article says matters to investors assessing OpenAI and Anthropic valuations.

Artificial Analysis says Chinese open-weight AI models now trail the overall leader by 12 points in its revised ranking, compared with a 3-point gap under the previous system. The firm recently changed its tests to better reflect how people use AI, keeping many of them secret so companies cannot train specifically for known questions. Under the new system, publicly released-weight models scored 32 points, while Anthropic’s Claude Opus 5.5, the overall leader at the time, scored 58. Moonshot’s Kimi K3.1 Pro was also included in the 12-point gap reported by the firm.

The ranking has renewed questions about whether lower-cost, freely available Chinese AI models pose as much competitive pressure as investors and businesses have feared. Chinese companies have released models that businesses can download and adapt, often at lower cost than American alternatives. That has raised concerns that OpenAI and Anthropic could struggle to charge premium prices for proprietary models. The article says the new results may strengthen the investment case for Anthropic’s planned IPO and a possible future OpenAI listing, while noting this is an implication of the rankings rather than a confirmed outcome.

Artificial Analysis CEO Sridhar Ramaswamy said the gap between open models and the leader expanded from 3 points to 12 when benchmarks became private, and asked whether earlier tests had been leaking. When firms know the tests in advance, they can train models to score well on those tasks without necessarily matching that performance in real-world use; Microsoft CEO Satya Nadella and other experts call this “benchmark hacking.” The revised results indicate a larger US lead in complex scientific tasks: even Anthropic’s smaller Claude Haiku 5.5 outperformed every model with publicly released weights. Trillium Labs cofounder Nathan Lambert said the earlier index had likely rated open-weight models too favorably.

Software developer OpenCode cofounder Dax Raad examined traffic through OpenRouter, a service that gives developers access to models from different providers. He estimated that almost all DeepSeek traffic from the previous day on OpenRouter could be handled by one rack of Nvidia computers costing roughly $6 million to $7 million. OpenRouter captures only part of the market, so those figures do not measure all demand for open-weight models. Raad argued that much of the sector’s infrastructure spending comes from major providers such as OpenAI and Anthropic, rather than smaller developers using open models. ChatGPT has more than 1 billion monthly users, whose activity may exceed what AI gateways such as OpenRouter capture.

The rankings and usage estimate leave open two questions: whether open-weight models match leading American models and whether demand for them is as large as the attention suggests. Lambert said they remain competitive in areas such as software development and can give businesses more control and potentially lower costs. The evidence challenges the claim that Chinese AI companies are rapidly catching US leaders and threatening their business models, a distinction the article says matters to investors assessing OpenAI and Anthropic valuations.