Qwen Team Publishes E-Commerce Agent Benchmark

Researchers from Alibaba’s Qwen team, Taobao & Tmall Group, and the Hong Kong University of Science and Technology (HKUST) published a paper on August 31, 2026, introducing E-Commerce Bench, a benchmark for measuring AI agents’ ability to run a business over the long term. Agents receive 100,000 yuan and must operate up to four stores for 365 days, aiming to maximize year-end assets. The test covers supplier negotiations, sourcing and pricing products, shipping, returns, restocking, and cash management.

The simulation includes 6,886 products across 60 categories and 576 suppliers, based on Taobao & Tmall data. Annual promotions, natural disasters, and supply-chain disruptions change market conditions. Unlike short tasks that reset, decisions carry over to subsequent days. Revenue from shipped orders is held for 9 days and must then be withdrawn through a separate action. Since purchasing and storage costs are charged upfront, excessive inventory can leave an agent short of cash.

To keep comparisons consistent, the benchmark uses fixed rules for customer purchases and supplier pricing decisions. A negotiation program determines suppliers’ asking prices, discounts, and whether deals are accepted or rejected. Large language models (AI systems that understand and generate text) express those decisions as natural conversation but do not change the price terms. This limits variation caused by counterpart models’ replies and prevents conversation from bypassing minimum supplier prices.

In the paper’s reported evaluation, researchers tested 18 models five times each. GPT-5.6 Sol had the highest average year-end assets, turning the initial 100,000 yuan into 1,431,425 yuan, about 14.3 times the starting amount. Among open-weight models, whose model weights are publicly available for use, Qwen3.8-Max-Preview ranked first with 416,252 yuan.

Despite leading in total assets, GPT-5.6 Sol ranked 16th among the 18 models in fraud avoidance. Fable5 scored higher on operating efficiency, measured as profit per tool call. The seven evaluation areas were total assets, negotiation quality, fraud avoidance, cash flow and solvency, operating efficiency, task completion, and long-term use of experience; no model led all seven. Ten of the 90 runs ended in bankruptcy. In one GPT-5.5 run, overspending on inventory depleted funds during January, before sales revenue became available. The research code and data were released under the Apache 2.0 license. The official project page lists results for 21 models, including evaluations added after publication; the experiments reported in the paper covered 18 models.

Researchers from Alibaba’s Qwen team, Taobao & Tmall Group, and the Hong Kong University of Science and Technology (HKUST) published a paper on August 31, 2026, introducing E-Commerce Bench, a benchmark for measuring AI agents’ ability to run a business over the long term. Agents receive 100,000 yuan and must operate up to four stores for 365 days, aiming to maximize year-end assets. The test covers supplier negotiations, sourcing and pricing products, shipping, returns, restocking, and cash management.

The simulation includes 6,886 products across 60 categories and 576 suppliers, based on Taobao & Tmall data. Annual promotions, natural disasters, and supply-chain disruptions change market conditions. Unlike short tasks that reset, decisions carry over to subsequent days. Revenue from shipped orders is held for 9 days and must then be withdrawn through a separate action. Since purchasing and storage costs are charged upfront, excessive inventory can leave an agent short of cash.

To keep comparisons consistent, the benchmark uses fixed rules for customer purchases and supplier pricing decisions. A negotiation program determines suppliers’ asking prices, discounts, and whether deals are accepted or rejected. Large language models (AI systems that understand and generate text) express those decisions as natural conversation but do not change the price terms. This limits variation caused by counterpart models’ replies and prevents conversation from bypassing minimum supplier prices.

In the paper’s reported evaluation, researchers tested 18 models five times each. GPT-5.6 Sol had the highest average year-end assets, turning the initial 100,000 yuan into 1,431,425 yuan, about 14.3 times the starting amount. Among open-weight models, whose model weights are publicly available for use, Qwen3.8-Max-Preview ranked first with 416,252 yuan.

Despite leading in total assets, GPT-5.6 Sol ranked 16th among the 18 models in fraud avoidance. Fable5 scored higher on operating efficiency, measured as profit per tool call. The seven evaluation areas were total assets, negotiation quality, fraud avoidance, cash flow and solvency, operating efficiency, task completion, and long-term use of experience; no model led all seven. Ten of the 90 runs ended in bankruptcy. In one GPT-5.5 run, overspending on inventory depleted funds during January, before sales revenue became available. The research code and data were released under the Apache 2.0 license. The official project page lists results for 21 models, including evaluations added after publication; the experiments reported in the paper covered 18 models.