ThinkingBox Turns Database Checks into RL Rewards

Microsoft and Hugging Face report that ThinkingBox-Training used executable checks of final database states as rewards to fine-tune Qwen3.8-27B on tasks outside ThinkingBox-Bench. After 50 GRPO steps, average success across 10,140 benchmark attempts rose from 51.70% to 60.78%, and tasks passed on all 20 attempts increased from 38 (7.50%) to 107 (21.10%). Tasks solved at least once fell from 453 (89.35%) to 446 (87.97%).

ThinkingBox-Bench contains 507 tasks. Each gives an agent a real backend and tools, then checks the resulting database records. The team selected training tasks using offline policy-relative difficulty: it probed each candidate with the base model, then kept a fixed pool of tasks the model solved sometimes, excluding tasks it always failed or passed. The benchmark itself was not used for training; its domains, databases and most tools were absent from the training pool, though the two sets shared the ThinkingBox framework, simulated users, executable grading, some tools and a similar task style.

For one hotel-booking task, the base model changed a booking and issued refunds without required manager approval in 15 of 20 attempts. After fine-tuning, it preserved the booking and payment state in all 20 attempts; all expected records matched in 19. Across five benchmark domains, pass@1 and observed 20/20 rose in every domain, while pass@20 increased in neobank and consulting and declined in retail, travel and auto insurance. Auto insurance had the largest pass@20 decline, −7.0 points; retail had the largest observed-20/20 gain, +24.5 points.

In the reference run, the team trained on 187 tasks. Each update generated 18 groups of 8 rollouts, or 144 episodes, across 24 H100 GPUs. Training used token-mean GRPO, a 1e-6 learning rate and zero weight decay for 50 steps. Episodes used fresh backends, isolated MCP sessions and GPT-5 Chat as the simulated user; most rewards came from deterministic database checks, while a minority of narrow rubric requirements could use GPT-5 Chat as a judge. Backend timeouts were treated as system errors and affected groups were refilled rather than scored as failures.

The estimated cost of a 507-task, 20-run evaluation campaign fell from $925.80 to $855.58, a 7.6% decrease. Cost per successful attempt dropped from $0.177 to $0.139 (21.4%), and estimated cost per task passed on all 20 attempts fell from $24.36 to $8.00 (67.2%). The authors describe these dollar figures as a comparative serving index, not an invoice or cloud bill; they include model inference but exclude training and policy-probe costs. They say the fine-tuned model reached $8.00 per dependable task, compared with $6.80 for GPT-5.4, $7.45 for GPT-6 Astra and $7.80 for Claude Opus 5.5.

Microsoft and Hugging Face report that ThinkingBox-Training used executable checks of final database states as rewards to fine-tune Qwen3.8-27B on tasks outside ThinkingBox-Bench. After 50 GRPO steps, average success across 10,140 benchmark attempts rose from 51.70% to 60.78%, and tasks passed on all 20 attempts increased from 38 (7.50%) to 107 (21.10%). Tasks solved at least once fell from 453 (89.35%) to 446 (87.97%).

ThinkingBox-Bench contains 507 tasks. Each gives an agent a real backend and tools, then checks the resulting database records. The team selected training tasks using offline policy-relative difficulty: it probed each candidate with the base model, then kept a fixed pool of tasks the model solved sometimes, excluding tasks it always failed or passed. The benchmark itself was not used for training; its domains, databases and most tools were absent from the training pool, though the two sets shared the ThinkingBox framework, simulated users, executable grading, some tools and a similar task style.

For one hotel-booking task, the base model changed a booking and issued refunds without required manager approval in 15 of 20 attempts. After fine-tuning, it preserved the booking and payment state in all 20 attempts; all expected records matched in 19. Across five benchmark domains, pass@1 and observed 20/20 rose in every domain, while pass@20 increased in neobank and consulting and declined in retail, travel and auto insurance. Auto insurance had the largest pass@20 decline, −7.0 points; retail had the largest observed-20/20 gain, +24.5 points.

In the reference run, the team trained on 187 tasks. Each update generated 18 groups of 8 rollouts, or 144 episodes, across 24 H100 GPUs. Training used token-mean GRPO, a 1e-6 learning rate and zero weight decay for 50 steps. Episodes used fresh backends, isolated MCP sessions and GPT-5 Chat as the simulated user; most rewards came from deterministic database checks, while a minority of narrow rubric requirements could use GPT-5 Chat as a judge. Backend timeouts were treated as system errors and affected groups were refilled rather than scored as failures.

The estimated cost of a 507-task, 20-run evaluation campaign fell from $925.80 to $855.58, a 7.6% decrease. Cost per successful attempt dropped from $0.177 to $0.139 (21.4%), and estimated cost per task passed on all 20 attempts fell from $24.36 to $8.00 (67.2%). The authors describe these dollar figures as a comparative serving index, not an invoice or cloud bill; they include model inference but exclude training and policy-probe costs. They say the fine-tuned model reached $8.00 per dependable task, compared with $6.80 for GPT-5.4, $7.45 for GPT-6 Astra and $7.80 for Claude Opus 5.5.