LMSYS Boosts Coding Agent Training Results

LMSYS trained Qwen3-30B-A3B-Instruct with fully asynchronous agentic reinforcement learning on AMD Instinct MI355X and raised its pass@1 score on a 100-task stratified SWE-bench Verified subset from 7.3% to 27.5%. The team used Miles, SGLang, Megatron-LM and Harbor in a ROCm container, and says the main bottlenecks were long-running trajectories, CPU sandboxes and data movement.

The training recipe had two stages: one epoch of supervised fine-tuning on about 4,000 successful Open-SWE-Traces agent runs, followed by asynchronous GRPO on about 1,350 SWE-smith tasks from 47 repositories absent from SWE-bench Verified. Each task had 8 attempts; a run earned a reward of 1 when repository tests passed and had no KL penalty. Evaluation used 4 attempts per task. Trajectories were capped at 8,192 generated tokens per turn, 65,536 tokens total and 80 turns.

In coding-agent training, each sample is an episode in which a model reads a GitHub issue, runs 20 to 100+ shell commands in a sandbox and receives a reward after tests. In GRPO, a group normally waits for its slowest attempt; two attempts at one task could differ 2.5× in length. In the team’s run, 84% of trajectories reached the turn limit, although successful runs used a median of 8 turns, and 32% of rollout time came after 90% of trajectories had finished. Capping trajectory length made training steps 3× faster without affecting successful runs.

The system separated inference and training across nodes, each with 8 MI355X GPUs. Eight single-GPU SGLang engines handled inference, while Megatron-LM trained the model and sent updated weights back; Miles let rollout and training overlap instead of waiting for every trajectory. A measured training step averaged 208 seconds: actor updates accounted for 63%, rollout waiting 29% and log-probability recomputation 8%, with weight synchronization logged separately.

Each trajectory used a CPU sandbox, so the team shifted these environments to E2B cloud microVMs with 2 vCPUs and 4 GB each. At 384 concurrent trajectories across 4 nodes, rollout throughput reached about 3,480 trajectories per hour, roughly 4× the local-Docker baseline of about 860. Sandbox startup took a median 1.9 seconds in the cloud, compared with 11.7 seconds for local Docker under load; cloud runs added 15–40 seconds per trajectory, or 4–11% of its time. LLM calls used 77% of cloud trajectory time, while tests, tool calls and sandbox setup used 22%. In steady state, 2.6% of trajectories failed and 0.09% of 300,500 tunnelled LLM calls returned an error.

The team found infrastructure faults could receive the same zero reward as an incorrect code fix: 3.1% of trajectories ended in infrastructure errors. Scaling also exposed fixed limits, including 31 Docker networks per machine, 100 image downloads per hour across the cluster, 100 agent connections in Miles and one CPU core handling agent requests. Removing concurrency caps, using a shared sandbox network, capping engine context at 64K tokens, pinning the NCCL interface and keeping HIP graphs enabled addressed reported bottlenecks. Without the context cap, some generations reached 113K–186K tokens and filled an engine’s 1.5M-token KV cache.

The base model scored 7.3% pass@1, supervised fine-tuning raised it to 12.8%, and agentic reinforcement learning added 14.7 percentage points to reach 27.5%. The run lasted 145 steps and resumed from checkpoints after 3 restarts. Separately, scaling inference from 1 to 3 8-GPU nodes, cloud sandboxes and removed concurrency limits produced the roughly 4× rollout throughput gain; this was a separate comparison from the 3× faster steps achieved by length capping on the same 8-GPU inference node. The team identified DeepSeek-V4-Flash and GLM-5.3 among models for future runs.

LMSYS trained Qwen3-30B-A3B-Instruct with fully asynchronous agentic reinforcement learning on AMD Instinct MI355X and raised its pass@1 score on a 100-task stratified SWE-bench Verified subset from 7.3% to 27.5%. The team used Miles, SGLang, Megatron-LM and Harbor in a ROCm container, and says the main bottlenecks were long-running trajectories, CPU sandboxes and data movement.

The training recipe had two stages: one epoch of supervised fine-tuning on about 4,000 successful Open-SWE-Traces agent runs, followed by asynchronous GRPO on about 1,350 SWE-smith tasks from 47 repositories absent from SWE-bench Verified. Each task had 8 attempts; a run earned a reward of 1 when repository tests passed and had no KL penalty. Evaluation used 4 attempts per task. Trajectories were capped at 8,192 generated tokens per turn, 65,536 tokens total and 80 turns.

In coding-agent training, each sample is an episode in which a model reads a GitHub issue, runs 20 to 100+ shell commands in a sandbox and receives a reward after tests. In GRPO, a group normally waits for its slowest attempt; two attempts at one task could differ 2.5× in length. In the team’s run, 84% of trajectories reached the turn limit, although successful runs used a median of 8 turns, and 32% of rollout time came after 90% of trajectories had finished. Capping trajectory length made training steps 3× faster without affecting successful runs.

The system separated inference and training across nodes, each with 8 MI355X GPUs. Eight single-GPU SGLang engines handled inference, while Megatron-LM trained the model and sent updated weights back; Miles let rollout and training overlap instead of waiting for every trajectory. A measured training step averaged 208 seconds: actor updates accounted for 63%, rollout waiting 29% and log-probability recomputation 8%, with weight synchronization logged separately.

Each trajectory used a CPU sandbox, so the team shifted these environments to E2B cloud microVMs with 2 vCPUs and 4 GB each. At 384 concurrent trajectories across 4 nodes, rollout throughput reached about 3,480 trajectories per hour, roughly 4× the local-Docker baseline of about 860. Sandbox startup took a median 1.9 seconds in the cloud, compared with 11.7 seconds for local Docker under load; cloud runs added 15–40 seconds per trajectory, or 4–11% of its time. LLM calls used 77% of cloud trajectory time, while tests, tool calls and sandbox setup used 22%. In steady state, 2.6% of trajectories failed and 0.09% of 300,500 tunnelled LLM calls returned an error.

The team found infrastructure faults could receive the same zero reward as an incorrect code fix: 3.1% of trajectories ended in infrastructure errors. Scaling also exposed fixed limits, including 31 Docker networks per machine, 100 image downloads per hour across the cluster, 100 agent connections in Miles and one CPU core handling agent requests. Removing concurrency caps, using a shared sandbox network, capping engine context at 64K tokens, pinning the NCCL interface and keeping HIP graphs enabled addressed reported bottlenecks. Without the context cap, some generations reached 113K–186K tokens and filled an engine’s 1.5M-token KV cache.

The base model scored 7.3% pass@1, supervised fine-tuning raised it to 12.8%, and agentic reinforcement learning added 14.7 percentage points to reach 27.5%. The run lasted 145 steps and resumed from checkpoints after 3 restarts. Separately, scaling inference from 1 to 3 8-GPU nodes, cloud sandboxes and removed concurrency limits produced the roughly 4× rollout throughput gain; this was a separate comparison from the 3× faster steps achieved by length capping on the same 8-GPU inference node. The team identified DeepSeek-V4-Flash and GLM-5.3 among models for future runs.