Vending-Bench 2

Citation frequency 2/3
What does it measure?
Tests how well an AI runs a simulated vending-machine business for a year: managing inventory, negotiating with suppliers by email, setting prices, and handling failed deliveries and refund demands. Made by Andon Labs.
Who evaluates it, and how?
The AI starts with $500 and is scored on the bank balance after a year, averaged over five runs per model.
Reading the score
Scores are the money (USD) left at the end of the simulation. Higher is better.
Top score$16,843
Models tested57
Scoring basisHigher is better
Model release date

Can it run a small business alone for a year and make money?

Vending-Bench 2

  1. 🥇GoogleGemini 4 Neon—$16,843—
  2. 🥈GoogleGemini 4 Carbon—$16,484—
  3. 🥉OpenAIGPT-6 Astra34.2%$15,51520.8%
  4. #4GoogleGemini 4 Hydrogen—$14,549—
  5. #5OpenAIGPT-6 Sol32.2%$14,428—
  6. #6GoogleGemini 4 Argon39.5%$13,71817.1%
  7. #7OpenAIGPT-6.1 Sol—$12,861—
  8. #8AnthropicClaude Opus 532.2%$11,182—
  9. #9AnthropicClaude Sonnet 5.5—$11,165—
  10. #10AnthropicClaude Opus 4.721.1%$10,937—
  11. #11SpaceXSIGrok 4.7—$10,537—
  12. #12AnthropicClaude Fable 5.2—$9,62841.3%
  13. #13OpenAIGPT-5.6 Sol30.6%$9,619—
  14. #14AnthropicClaude Opus 5.538.2%$9,23521.3%
  15. #15OpenAIGPT-6.1 Terra—$9,107—
  16. #16SpaceXSIGrok 4.6—$9,047—
  17. #17Z.aiGLM 5.220.4%$8,314—
  18. #18Z.aiGLM 5.3—$8,164—
  19. #19AnthropicClaude Opus 4.6—$8,0184.2%
  20. #20OpenAIGPT-5.526.6%$7,5246.3%
  21. #21OpenAIGPT-5.6 Terra28.0%$7,343—
  22. #22AnthropicClaude Sonnet 4.6—$7,204—
  23. #23MetaMuse Spark 1.1—$6,520—
  24. #24AnthropicClaude Sonnet 5—$6,378—
  25. #25Moonshot AIKimi K2.69.2%$6,205—
  26. #26OpenAIGPT-5.420.5%$6,144—
  27. #27AnthropicClaude Opus 4.827.0%$5,7878.3%
  28. #28AnthropicClaude Fable 5Reasoning effort: High—$5,68016.1%
  29. #29Z.aiGLM-5.111.5%$5,634—
  30. #30AnthropicClaude Fable 5.1—$5,42217.9%
  31. #31GoogleGemini 3.5 Flash—$5,396—
  32. #32Moonshot AIKimi K328.3%$5,165—
  33. #33AlibabaQwen3.6 Plus8.6%$5,115—
  34. #34GoogleGemini 3.8 Flash—$5,0945.8%
  35. #35Moonshot AIKimi K2.7 Code—$5,083—
  36. #36AnthropicClaude Opus 4.5—$4,9673.8%
  37. #37SpaceXSIGrok 4.20 (Reasoning)—$4,663—
  38. #38Z.aiGLM-5—$4,432—
  39. #39AlibabaQwen3.6 Max—$4,254—
  40. #40OpenAIGPT-5.6 Luna30.3%$4,095—
  41. #41SpaceXSIGrok 4.527.0%$3,887—
  42. #42AnthropicClaude Sonnet 4.5—$3,8392.1%
  43. #43GoogleGemini 3.1 Pro16.4%$3,774—
  44. #44GoogleGemini 3 Flash—$3,635—
  45. #45AnthropicClaude Haiku 5.5—$3,602—
  46. #46DeepSeekDeepSeek V4 Pro12.4%$3,285—
  47. #47MiniMaxMiniMax M3—$2,158—
  48. #48Moonshot AIKimi K2.5—$1,198—
  49. #49SpaceXSIGrok 4.1 Fast (Reasoning)—$1,107—
  50. #50GoogleGemini 2.5 Pro—$574—
  51. #51GoogleGemini 2.5 Flash—$549—
  52. #52AnthropicClaude Haiku 4.5—$459—
  53. #53SpaceXSIGrok 4.37.2%$35—
  54. #54AlibabaQwen3.5 Plus—$1—
  55. #55OpenAIGPT OSS 120B—−$22—
  56. #56MiniMaxMiniMax M2.5—−$23—
  57. #57OpenAIGPT-5 Mini—−$31—
Half of models ≤ $5,634

The "harness" is the agent program the AI used to carry out the task. The same model can score very differently depending on its harness and reasoning effort.