OpenRouter Rebuilds Provider Routing

OpenRouter announced on October 28, 2026, its largest provider-routing algorithm improvement since the company was founded. The new default router weighs effective price, response speed, throughput, reliability and response quality instead of routing mainly by input-token price. OpenRouter says users do not need to change their code to use it. In an experiment, the new system reduced median time to first token by 16.9%, increased throughput by 16.3% and cut capacity errors by 67%, while cost per 1,000 requests rose 6.0%.

About 77% of OpenRouter requests go to models served by multiple providers. Under the former approach, cheaper providers received more traffic and failed requests moved to the next-cheapest option. OpenRouter says that system could make users wait through failures, overlooked real-time performance changes and rewarded price cuts over infrastructure investment. Providers could also offset a low input-token price with higher charges for cached or completion tokens.

The new router estimates input and output token use from recent model traffic and calculates each provider’s effective price using measured traffic, including cache hits and the average input/output mix. It also scores throughput, measured in output tokens per second across the full response, and time to first token (TTFT; the wait before streaming begins). Rate limits and provider-caused failures add penalties because a failed request costs time and may require a retry. The router also checks whether responses are usable, including whether tool calls are properly formed; providers flagged for broken tool calls or benchmark accuracy far below the model’s expected score receive only small amounts of test traffic until the flag clears.

Traffic is shared among providers near the top of the router’s ranking, with share falling quickly beyond a competitive edge. Guardrails keep a small flow of requests going to every provider, reduce traffic to providers that fail or hit rate limits, and cap traffic at the load each provider can handle before returning HTTP 429 errors. New endpoints begin with typical figures for their model. Conversations stay with their initially selected provider while its prompt cache remains warm, but can fall back if it becomes unavailable; failed requests are retried with the next-ranked provider. Explicit provider.order, provider.sort, and certain provider filters affect whether the new routing logic applies.

OpenRouter tested the change in a one-week switchback experiment across a representative sample of open-source models. It alternated matched three-hour blocks between the old and new routers, discarded one hour of data after each changeover, and split more than 24 million requests containing 103 billion input tokens between the systems. Median TTFT fell from 1,007 ms to 837 ms, while p95 TTFT fell from 4,601 ms to 3,733 ms. Median throughput rose from 62.4 to 72.6 tokens per second. Median whole-request time fell from 4.00 to 3.24 seconds, and p95 time from 41.5 to 34.9 seconds. Requests encountering at least one HTTP 429 declined from 7.1% to 3.8%, reported as 67% fewer; requests needing a retry fell from 10.4% to 6.9%, or 47% fewer.

The experiment also found cost per 1,000 requests rose from $1.25 to $1.32, or 6.0%, and cost per million tokens rose from $0.301 to $0.324, or 7.3%. Time spent in failed attempts fell from 349 ms to 242 ms per request, a reported 55% reduction. Users who want price-only routing can select price sorting, append :floor to a model slug, or change the default in account settings; users can also specify provider order or limit the eligible providers.

OpenRouter announced on October 28, 2026, its largest provider-routing algorithm improvement since the company was founded. The new default router weighs effective price, response speed, throughput, reliability and response quality instead of routing mainly by input-token price. OpenRouter says users do not need to change their code to use it. In an experiment, the new system reduced median time to first token by 16.9%, increased throughput by 16.3% and cut capacity errors by 67%, while cost per 1,000 requests rose 6.0%.

About 77% of OpenRouter requests go to models served by multiple providers. Under the former approach, cheaper providers received more traffic and failed requests moved to the next-cheapest option. OpenRouter says that system could make users wait through failures, overlooked real-time performance changes and rewarded price cuts over infrastructure investment. Providers could also offset a low input-token price with higher charges for cached or completion tokens.

The new router estimates input and output token use from recent model traffic and calculates each provider’s effective price using measured traffic, including cache hits and the average input/output mix. It also scores throughput, measured in output tokens per second across the full response, and time to first token (TTFT; the wait before streaming begins). Rate limits and provider-caused failures add penalties because a failed request costs time and may require a retry. The router also checks whether responses are usable, including whether tool calls are properly formed; providers flagged for broken tool calls or benchmark accuracy far below the model’s expected score receive only small amounts of test traffic until the flag clears.

Traffic is shared among providers near the top of the router’s ranking, with share falling quickly beyond a competitive edge. Guardrails keep a small flow of requests going to every provider, reduce traffic to providers that fail or hit rate limits, and cap traffic at the load each provider can handle before returning HTTP 429 errors. New endpoints begin with typical figures for their model. Conversations stay with their initially selected provider while its prompt cache remains warm, but can fall back if it becomes unavailable; failed requests are retried with the next-ranked provider. Explicit provider.order, provider.sort, and certain provider filters affect whether the new routing logic applies.

OpenRouter tested the change in a one-week switchback experiment across a representative sample of open-source models. It alternated matched three-hour blocks between the old and new routers, discarded one hour of data after each changeover, and split more than 24 million requests containing 103 billion input tokens between the systems. Median TTFT fell from 1,007 ms to 837 ms, while p95 TTFT fell from 4,601 ms to 3,733 ms. Median throughput rose from 62.4 to 72.6 tokens per second. Median whole-request time fell from 4.00 to 3.24 seconds, and p95 time from 41.5 to 34.9 seconds. Requests encountering at least one HTTP 429 declined from 7.1% to 3.8%, reported as 67% fewer; requests needing a retry fell from 10.4% to 6.9%, or 47% fewer.

The experiment also found cost per 1,000 requests rose from $1.25 to $1.32, or 6.0%, and cost per million tokens rose from $0.301 to $0.324, or 7.3%. Time spent in failed attempts fell from 349 ms to 242 ms per request, a reported 55% reduction. Users who want price-only routing can select price sorting, append :floor to a model slug, or change the default in account settings; users can also specify provider order or limit the eligible providers.