Manifest said it no longer believes in model routing for most use cases after launching its LLM router in March, deprecating it in June, and scheduling a full shutdown for September 1st. The company said a single battle-tested model is usually the better choice, despite recent launches promising lower inference costs through routers that pick a model on the fly.

Manifest’s router, part of its LLM gateway, classified each request into four tiers: simple, standard, complex and reasoning. After four months of usage across 7000 cloud users, the company reported mixed results, GitHub issues and discussions, and concluded that prompt text alone often cannot reveal task complexity because tool calls, web searches and repository context may change the work required.

Manifest said caching often reduced costs more reliably than routing because cache reads are between 75% and 90% cheaper than uncached inputs. The company said system prompts and conversation history can represent many tokens, and prefix cache works well when repeated text sits at the beginning of the prompt; a cache-aware router would add stickiness to the first model and keep querying it, effectively avoiding routing.

Manifest also said LLM routers can reduce behavior consistency by moving between models during working sessions. The company argued engineers should choose models and effort parameters based on intent, and said unpredictability in automated agentic workflows or autonomous agents can add maintenance costs across evals, system prompts and observability. Manifest concluded that routing was not worth it for most use cases it saw because savings were paid elsewhere in harder-to-estimate costs.

Manifest said it no longer believes in model routing for most use cases after launching its LLM router in March, deprecating it in June, and scheduling a full shutdown for September 1st. The company said a single battle-tested model is usually the better choice, despite recent launches promising lower inference costs through routers that pick a model on the fly.

Manifest’s router, part of its LLM gateway, classified each request into four tiers: simple, standard, complex and reasoning. After four months of usage across 7000 cloud users, the company reported mixed results, GitHub issues and discussions, and concluded that prompt text alone often cannot reveal task complexity because tool calls, web searches and repository context may change the work required.

Manifest said caching often reduced costs more reliably than routing because cache reads are between 75% and 90% cheaper than uncached inputs. The company said system prompts and conversation history can represent many tokens, and prefix cache works well when repeated text sits at the beginning of the prompt; a cache-aware router would add stickiness to the first model and keep querying it, effectively avoiding routing.

Manifest also said LLM routers can reduce behavior consistency by moving between models during working sessions. The company argued engineers should choose models and effort parameters based on intent, and said unpredictability in automated agentic workflows or autonomous agents can add maintenance costs across evals, system prompts and observability. Manifest concluded that routing was not worth it for most use cases it saw because savings were paid elsewhere in harder-to-estimate costs.