LongCat-2.5 Targets Multimodal Agentic Tasks

LongCat introduced LongCat-2.5 on November 1, 2026, describing it as a native multimodal model for long-horizon agentic tasks. It has 1.6 trillion total parameters and approximately 48 billion activated per token. The company says it builds on LongCat-2.0 with stronger visual understanding and agentic coding, and is trained on long sequences and large-scale reinforcement learning for complex codebase development and full-project builds.

The model represents visual signals as multi-level discrete tokens, allowing image and text inputs to use a shared token-based format. Its design combines native-resolution visual tokenization, offline preprocessing and visual embeddings learned jointly with the backbone. LongCat says this supports visual coding, creative design and 3D tasks. A smaller mixture-of-experts model averaging approximately 3B activated parameters showed interwoven visual and textual embeddings in t-SNE visualizations, which the company presents as preliminary validation of a shared semantic space.

LongCat describes a training pipeline that synthesizes coding tasks, interactive environments and verifiable tests. It filters GitHub pull requests and linked issues, then uses agents to annotate tasks and create environments, with fail-to-pass and pass-to-pass tests checking solutions. Real internal coding tasks and evaluation criteria add coverage of everyday engineering work; rollout pass rates guide relabeling, while analysis of false positives and false negatives updates tests. Training also mixes tasks and scaffolds from different benchmarks, which LongCat says improves generalization across tasks.

To make reinforcement learning more efficient, the system uses asynchronous partial rollouts: it collects an oversampled batch, interrupts and saves unfinished trajectories, then resumes them in a later sampling phase. It adjusts oversampling using precomputed pass rates and filters groups where every rollout succeeds or fails. Because a trajectory can use outputs from different historical checkpoints, the team clips differences between sampling-policy and current-policy log probabilities and rescales token contributions to limit bias from stale samples. LongCat says it disables network access in task environments, removes Git history and cached files, and uses an LLM judge to audit trajectories for exploits. It also penalizes repetitive reasoning and tool calls and redistributes rewards among successful samples to address overthinking and rapidly growing response length.

LongCat-2.5 retains expert groups from LongCat-2.0 for agent capabilities, reasoning and interaction, including multi-turn task execution, single-turn reasoning, detailed instruction following and safety. During multi-teacher training, the student samples tasks online and domain experts train it using full-vocabulary KL divergence. The team splits teacher responses into fixed-length segments to create prefixes, limits student self-sampling to the same length, and groups prefixes from a teacher trajectory on one inference engine to share prefill computation and KV cache. Tree Attention processes teacher trajectories and student samples together in one forward and backward pass.

Long-context training runs on domestic AI accelerators with less than 80 GB of memory per accelerator. LongCat aligns parallel groups with the supernode network layout and uses hybrid vocabulary parallelism to reduce memory use. Its dynamic expert replication activates only after expert-parallel load imbalance passes a configured threshold. Dual-shard FSDP distributes model parameters within each supernode and further shards FP32 master weights and optimizer states across FSDP domains. The company reports 86% higher per-device throughput than a DDP baseline on long-sequence supervised fine-tuning tasks.

LongCat says its training and inference systems preserve bitwise-identical per-token log probabilities while allowing different parallelism configurations, with near-zero performance overhead. For long-context agent rollouts, it uses separate prefill and decode configurations, cache management, cross-layer prefetching and capacity-aware scheduling. In the tested workload, a 3:2 prefill-to-decode instance ratio performed best. The article also describes a W4A4 inference solution for the 1.6-trillion-parameter mixture-of-experts model, jointly optimizing quantization accuracy, low-level kernels and inference pipelines; the source endś

LongCat introduced LongCat-2.5 on November 1, 2026, describing it as a native multimodal model for long-horizon agentic tasks. It has 1.6 trillion total parameters and approximately 48 billion activated per token. The company says it builds on LongCat-2.0 with stronger visual understanding and agentic coding, and is trained on long sequences and large-scale reinforcement learning for complex codebase development and full-project builds.

The model represents visual signals as multi-level discrete tokens, allowing image and text inputs to use a shared token-based format. Its design combines native-resolution visual tokenization, offline preprocessing and visual embeddings learned jointly with the backbone. LongCat says this supports visual coding, creative design and 3D tasks. A smaller mixture-of-experts model averaging approximately 3B activated parameters showed interwoven visual and textual embeddings in t-SNE visualizations, which the company presents as preliminary validation of a shared semantic space.

LongCat describes a training pipeline that synthesizes coding tasks, interactive environments and verifiable tests. It filters GitHub pull requests and linked issues, then uses agents to annotate tasks and create environments, with fail-to-pass and pass-to-pass tests checking solutions. Real internal coding tasks and evaluation criteria add coverage of everyday engineering work; rollout pass rates guide relabeling, while analysis of false positives and false negatives updates tests. Training also mixes tasks and scaffolds from different benchmarks, which LongCat says improves generalization across tasks.

To make reinforcement learning more efficient, the system uses asynchronous partial rollouts: it collects an oversampled batch, interrupts and saves unfinished trajectories, then resumes them in a later sampling phase. It adjusts oversampling using precomputed pass rates and filters groups where every rollout succeeds or fails. Because a trajectory can use outputs from different historical checkpoints, the team clips differences between sampling-policy and current-policy log probabilities and rescales token contributions to limit bias from stale samples. LongCat says it disables network access in task environments, removes Git history and cached files, and uses an LLM judge to audit trajectories for exploits. It also penalizes repetitive reasoning and tool calls and redistributes rewards among successful samples to address overthinking and rapidly growing response length.

LongCat-2.5 retains expert groups from LongCat-2.0 for agent capabilities, reasoning and interaction, including multi-turn task execution, single-turn reasoning, detailed instruction following and safety. During multi-teacher training, the student samples tasks online and domain experts train it using full-vocabulary KL divergence. The team splits teacher responses into fixed-length segments to create prefixes, limits student self-sampling to the same length, and groups prefixes from a teacher trajectory on one inference engine to share prefill computation and KV cache. Tree Attention processes teacher trajectories and student samples together in one forward and backward pass.

Long-context training runs on domestic AI accelerators with less than 80 GB of memory per accelerator. LongCat aligns parallel groups with the supernode network layout and uses hybrid vocabulary parallelism to reduce memory use. Its dynamic expert replication activates only after expert-parallel load imbalance passes a configured threshold. Dual-shard FSDP distributes model parameters within each supernode and further shards FP32 master weights and optimizer states across FSDP domains. The company reports 86% higher per-device throughput than a DDP baseline on long-sequence supervised fine-tuning tasks.

LongCat says its training and inference systems preserve bitwise-identical per-token log probabilities while allowing different parallelism configurations, with near-zero performance overhead. For long-context agent rollouts, it uses separate prefill and decode configurations, cache management, cross-layer prefetching and capacity-aware scheduling. In the tested workload, a 3:2 prefill-to-decode instance ratio performed best. The article also describes a W4A4 inference solution for the 1.6-trillion-parameter mixture-of-experts model, jointly optimizing quantization accuracy, low-level kernels and inference pipelines; the source endś