FusionRoute addresses the efficiency-performance trade-off in large language models by moving beyond traditional sequence-level routing. Instead of relying on a single general-purpose model, it implements a token-level collaboration strategy using a lightweight router. This router performs two specific tasks at each decoding step: selecting the most appropriate domain expert and generating a complementary logit to refine the distribution.

By adding this logit to the expert's output, FusionRoute corrects the token distribution and expands the policy class of the model ensemble. Theoretical analysis demonstrates that augmenting expert selection with a trainable generator overcomes the limitations of pure expert-only routing, which is often restricted by global coverage assumptions. This allows the framework to recover optimal value functions under mild conditions while improving the accuracy of individual tokens.

Empirical evaluations using Llama-3 and Gemma-2 show FusionRoute outperforming model merging and fine-tuning across complex math and coding tasks. The system maintains high efficiency while achieving competitive results against sequence-level collaboration models. This approach marks a significant advancement in multi-LLM orchestration, offering a more robust and scalable solution for specialized AI reasoning and high-performance inference.

FusionRoute addresses the efficiency-performance trade-off in large language models by moving beyond traditional sequence-level routing. Instead of relying on a single general-purpose model, it implements a token-level collaboration strategy using a lightweight router. This router performs two specific tasks at each decoding step: selecting the most appropriate domain expert and generating a complementary logit to refine the distribution.

By adding this logit to the expert's output, FusionRoute corrects the token distribution and expands the policy class of the model ensemble. Theoretical analysis demonstrates that augmenting expert selection with a trainable generator overcomes the limitations of pure expert-only routing, which is often restricted by global coverage assumptions. This allows the framework to recover optimal value functions under mild conditions while improving the accuracy of individual tokens.

Empirical evaluations using Llama-3 and Gemma-2 show FusionRoute outperforming model merging and fine-tuning across complex math and coding tasks. The system maintains high efficiency while achieving competitive results against sequence-level collaboration models. This approach marks a significant advancement in multi-LLM orchestration, offering a more robust and scalable solution for specialized AI reasoning and high-performance inference.