Benchmarks/AIB DecisionBench-Game · game-0.5

Korean & Japanese judgment ranking Agent-role ranking

Which decision AI wins at a real-time game?

AIB DecisionBench-Game measures the accuracy, speed and cost of decision AI through real-time games. It is designed around how decision AI is used in real work: making fast, accurate decisions one after another.

22
Models: 4
Games: 616
Plays: 27,281
Real-time decisions

Design intent

Decision AI models pick one of the given options, with a probability, in 0.2–0.6 seconds. In real work these decisions come one after another, and the situation often won't wait for a decision to finish. Accuracy alone can't show this practical ability, so we measure it with real-time games, where even a slightly late decision decides the outcome.

How it is measured

Models measured together start the game at the same moment from the same starting conditions; models added later are measured separately under the same conditions. The game runs in real time, so even a correct decision doesn't count if it comes too late. So that one lucky or unlucky round doesn't decide the result, each game is played several times on different boards and averaged (whack-a-mole 10, snake 5, Tetris 5, Pong 8). Each model then plays the same game again with the game clock paused while it decides, which shows whether a low score comes from judgment or from speed.

Metrics

Real-time score — 0–100 per game; reflects judgment and speed together
Score with enough time — judgment alone
Points lost to slowness — the gap between the real-time score and the score with enough time
Response time · cost per 1,000 — how long one decision takes, and what 1,000 decisions cost
Games — whack-a-mole, snake, Tetris and Pong, with more to come

Measured October 12 – 24, 2026: overall #1 of 22 decision AI models is GPT-6 Luna Decisions (69.5).

Without a time limit, game rankings nearly match text judgment rankings; in real time they change a lot.
In Whack-a-mole, Snake, Tetris, Pong, occasional long delays hurt rankings more than typical speed did.
Judgment decides the ranking in Tetris, where scores stay spread out even without a time limit; speed decides it in Whack-a-mole, Snake, Pong, where most models score high without one.

See all insights

Overall ranking

🥇OpenAIGPT-6 Luna Decisions69.587.2
Whack-a-mole
93.6
Snake
71.0
Tetris
52.1
Pong
61.4
Median response
234ms
Cost / 1K
$0.041
🥈TypeSafeJev 1.1358.291.6
Whack-a-mole
82.7
Snake
50.4
Tetris
51.1
Pong
48.5
Median response
397ms
Cost / 1K
$0.030
🥉PerplexityDecider V1.256.486.6
Whack-a-mole
86.1
Snake
49.3
Tetris
28.8
Pong
61.4
Median response
349ms
Cost / 1K
$0.0075
#4PerplexityDecider Flash V1.255.580.5
Whack-a-mole
93.2
Snake
56.8
Tetris
20.1
Pong
52.1
Median response
269ms
Cost / 1K
$0.0032
#5CloudflareClef Omni55.371.7
Whack-a-mole
98.1
Snake
64.6
Tetris
12.3
Pong
46.4
Median response
184ms
Cost / 1K
$0.053
#6TypeSafeJev 1.1454.189.5
Whack-a-mole
80.7
Snake
45.7
Tetris
40.4
Pong
49.4
Median response
395ms
Cost / 1K
$0.028
#7PerplexityPerplexity Decider v1.1 27B52.791.8
Whack-a-mole
79.4
Snake
32.4
Tetris
42.0
Pong
56.8
Median response
391ms
Cost / 1K
$0.0087
#8CloudflareClef50.483.4
Whack-a-mole
73.0
Snake
50.1
Tetris
29.1
Pong
49.5
Median response
385ms
Cost / 1K
$0.113
#9Liquid AID142.587.2
Whack-a-mole
60.5
Snake
34.8
Tetris
31.5
Pong
43.1
Median response
514ms
Cost / 1K
$0.0078
#10CloudflareClef Flash41.977.4
Whack-a-mole
61.1
Snake
44.6
Tetris
14.1
Pong
47.9
Median response
410ms
Cost / 1K
$0.033
#11Liquid AID1 Ultra38.188.4
Whack-a-mole
52.7
Snake
21.5
Tetris
26.5
Pong
51.5
Median response
547ms
Cost / 1K
$0.014
#12Nace AIDrex Max33.981.3
Whack-a-mole
39.9
Snake
52.3
Tetris
23.3
Pong
20.3
Median response
710ms
Cost / 1K
$0.022
#13Nace AIDrex v1.531.481.9
Whack-a-mole
49.1
Snake
34.1
Tetris
18.3
Pong
24.1
Median response
664ms
Cost / 1K
$0.011
#14ScaleDownClassify-131.269.4
Whack-a-mole
49.5
Snake
42.9
Tetris
17.7
Pong
14.6
Median response
739ms
Cost / 1K
$0.039
#15MicrosoftMicrosoft-Decision-130.687.5
Whack-a-mole
30.5
Snake
38.6
Tetris
26.8
Pong
26.6
Median response
642ms
Cost / 1K
$0.014
#16AlibabaDecision Model Preview28.262.7
Whack-a-mole
45.5
Snake
24.7
Tetris
13.2
Pong
29.6
Median response
663ms
Cost / 1K
Free for now
#17Nace AIDrex v2.026.171.4
Whack-a-mole
40.7
Snake
28.8
Tetris
24.5
Pong
10.4
Median response
713ms
Cost / 1K
$0.018
#18UpstageSolar Decide Flash22.744.9
Whack-a-mole
36.7
Snake
23.3
Tetris
9.3
Pong
21.4
Median response
546ms
Cost / 1K
$0.027
#19Jared PalmerKev 4B22.674.6
Whack-a-mole
30.4
Snake
24.0
Tetris
14.4
Pong
21.4
Median response
747ms
Cost / 1K
$0.012
#20RespanSpan-0221.082.4
Whack-a-mole
33.8
Snake
18.1
Tetris
23.2
Pong
8.9
Median response
801ms
Cost / 1K
$0.0036
#21InceptionMercury Decide11.182.2
Whack-a-mole
15.1
Snake
16.2
Tetris
10.9
Pong
2.1
Median response
326ms
Cost / 1K
Free for now
#22UpstageSolar Decide6.255.6
Whack-a-mole
10.2
Snake
8.4
Tetris
—
Pong
0.0
Median response
5.8s
Cost / 1K
$0.022

Rank	Model	Overall score	Whack-a-mole	Snake	Tetris	Pong	Median response	Cost / 1K
🥇	OpenAIGPT-6 Luna Decisions	69.587.2	93.6	71.0	52.1	61.4	234ms	$0.041
🥈	TypeSafeJev 1.13	58.291.6	82.7	50.4	51.1	48.5	397ms	$0.030
🥉	PerplexityDecider V1.2	56.486.6	86.1	49.3	28.8	61.4	349ms	$0.0075
#4	PerplexityDecider Flash V1.2	55.580.5	93.2	56.8	20.1	52.1	269ms	$0.0032
#5	CloudflareClef Omni	55.371.7	98.1	64.6	12.3	46.4	184ms	$0.053
#6	TypeSafeJev 1.14	54.189.5	80.7	45.7	40.4	49.4	395ms	$0.028
#7	PerplexityPerplexity Decider v1.1 27B	52.791.8	79.4	32.4	42.0	56.8	391ms	$0.0087
#8	CloudflareClef	50.483.4	73.0	50.1	29.1	49.5	385ms	$0.113
#9	Liquid AID1	42.587.2	60.5	34.8	31.5	43.1	514ms	$0.0078
#10	CloudflareClef Flash	41.977.4	61.1	44.6	14.1	47.9	410ms	$0.033
#11	Liquid AID1 Ultra	38.188.4	52.7	21.5	26.5	51.5	547ms	$0.014
#12	Nace AIDrex Max	33.981.3	39.9	52.3	23.3	20.3	710ms	$0.022
#13	Nace AIDrex v1.5	31.481.9	49.1	34.1	18.3	24.1	664ms	$0.011
#14	ScaleDownClassify-1	31.269.4	49.5	42.9	17.7	14.6	739ms	$0.039
#15	MicrosoftMicrosoft-Decision-1	30.687.5	30.5	38.6	26.8	26.6	642ms	$0.014
#16	AlibabaDecision Model Preview	28.262.7	45.5	24.7	13.2	29.6	663ms	Free for now
#17	Nace AIDrex v2.0	26.171.4	40.7	28.8	24.5	10.4	713ms	$0.018
#18	UpstageSolar Decide Flash	22.744.9	36.7	23.3	9.3	21.4	546ms	$0.027
#19	Jared PalmerKev 4B	22.674.6	30.4	24.0	14.4	21.4	747ms	$0.012
#20	RespanSpan-02	21.082.4	33.8	18.1	23.2	8.9	801ms	$0.0036
#21	InceptionMercury Decide	11.182.2	15.1	16.2	10.9	2.1	326ms	Free for now
#22	UpstageSolar Decide	6.255.6	10.2	8.4	—	0.0	5.8s	$0.022

The overall score averages each game's real-time score on a 0–100 scale. Ties are broken by cost per decision (differences under 10% count as equal), then median response time.

The gray number under each overall score is the overall score with enough time: the average of the same games replayed with the game clock paused, reflecting decision quality alone.

Results by game

Whack-a-mole

average of 10 runs

🐭, 🐰 and 🐹 pop up in a 3×3 grid for 0.4–1.9 seconds. Choosing 🐭 scores +2; choosing 🐰 or 🐹 scores −1. Each time, the AI model picks one cell or chooses none, and the game lasts 25 seconds.

🥇CloudflareClef Omni98.1±2.1
With enough time
100.0
Lost to slowness
−1.9
🐭 hit
98%
Median response
129ms
Slowest 5%
263ms
Cost / 1K
$0.032
🥈OpenAIGPT-6 Luna Decisions93.6±8.2
With enough time
99.5
Lost to slowness
−5.9
🐭 hit
94%
Median response
226ms
Slowest 5%
368ms
Cost / 1K
$0.020
🥉PerplexityDecider Flash V1.293.2±4.0
With enough time
100.0
Lost to slowness
−6.8
🐭 hit
93%
Median response
260ms
Slowest 5%
353ms
Cost / 1K
$0.0018
#4PerplexityDecider V1.286.1±4.1
With enough time
100.0
Lost to slowness
−13.9
🐭 hit
86%
Median response
336ms
Slowest 5%
438ms
Cost / 1K
$0.0035
#5TypeSafeJev 1.1382.7±5.6
With enough time
100.0
Lost to slowness
−17.3
🐭 hit
82%
Median response
384ms
Slowest 5%
503ms
Cost / 1K
$0.017
#6TypeSafeJev 1.1480.7±5.9
With enough time
100.0
Lost to slowness
−19.3
🐭 hit
81%
Median response
399ms
Slowest 5%
459ms
Cost / 1K
$0.018
#7PerplexityPerplexity Decider v1.1 27B79.4±4.7
With enough time
100.0
Lost to slowness
−20.6
🐭 hit
79%
Median response
406ms
Slowest 5%
579ms
Cost / 1K
$0.0036
#8CloudflareClef73.0±9.7
With enough time
100.0
Lost to slowness
−27.0
🐭 hit
72%
Median response
400ms
Slowest 5%
817ms
Cost / 1K
$0.053
#9CloudflareClef Flash61.1±8.8
With enough time
100.0
Lost to slowness
−38.9
🐭 hit
61%
Median response
475ms
Slowest 5%
971ms
Cost / 1K
$0.020
#10Liquid AID160.5±9.8
With enough time
100.0
Lost to slowness
−39.5
🐭 hit
60%
Median response
499ms
Slowest 5%
732ms
Cost / 1K
$0.0028
#11Liquid AID1 Ultra52.7±10.1
With enough time
100.0
Lost to slowness
−47.3
🐭 hit
52%
Median response
525ms
Slowest 5%
1.1s
Cost / 1K
$0.0058
#12ScaleDownClassify-149.5±9.5
With enough time
97.1
Lost to slowness
−47.5
🐭 hit
49%
Median response
551ms
Slowest 5%
861ms
Cost / 1K
$0.013
#13Nace AIDrex v1.549.1±8.5
With enough time
100.0
Lost to slowness
−50.9
🐭 hit
49%
Median response
504ms
Slowest 5%
950ms
Cost / 1K
$0.0043
#14AlibabaDecision Model Preview45.5±11.3
With enough time
66.1
Lost to slowness
−20.6
🐭 hit
53%
Median response
643ms
Slowest 5%
823ms
Cost / 1K
Free for now
#15Nace AIDrex v2.040.7±9.5
With enough time
100.0
Lost to slowness
−59.3
🐭 hit
40%
Median response
547ms
Slowest 5%
1.0s
Cost / 1K
$0.0065
#16Nace AIDrex Max39.9±7.4
With enough time
100.0
Lost to slowness
−60.1
🐭 hit
40%
Median response
538ms
Slowest 5%
1.1s
Cost / 1K
$0.011
#17UpstageSolar Decide Flash36.7±6.3
With enough time
50.4
Lost to slowness
−13.7
🐭 hit
49%
Median response
545ms
Slowest 5%
858ms
Cost / 1K
$0.021
#18RespanSpan-0233.8±9.3
With enough time
100.0
Lost to slowness
−66.2
🐭 hit
34%
Median response
763ms
Slowest 5%
1.1s
Cost / 1K
$0.0018
#19MicrosoftMicrosoft-Decision-130.5±6.9
With enough time
100.0
Lost to slowness
−69.5
🐭 hit
30%
Median response
631ms
Slowest 5%
1.6s
Cost / 1K
$0.0050
#20Jared PalmerKev 4B30.4±5.0
With enough time
100.0
Lost to slowness
−69.6
🐭 hit
31%
Median response
723ms
Slowest 5%
1.2s
Cost / 1K
$0.0046
#21InceptionMercury Decide15.1±4.2
With enough time
100.0
Lost to slowness
−84.9
🐭 hit
15%
Median response
302ms
Slowest 5%
834ms
Cost / 1K
Free for now
#22UpstageSolar Decide10.2±6.8
With enough time
97.0
Lost to slowness
−86.8
🐭 hit
9%
Median response
1.9s
Slowest 5%
7.2s
Cost / 1K
$0.021

Rank	Model	Real-time score	With enough time	Lost to slowness	🐭 hit	Median response	Slowest 5%	Cost / 1K
🥇	CloudflareClef Omni	98.1±2.1	100.0	−1.9	98%	129ms	263ms	$0.032
🥈	OpenAIGPT-6 Luna Decisions	93.6±8.2	99.5	−5.9	94%	226ms	368ms	$0.020
🥉	PerplexityDecider Flash V1.2	93.2±4.0	100.0	−6.8	93%	260ms	353ms	$0.0018
#4	PerplexityDecider V1.2	86.1±4.1	100.0	−13.9	86%	336ms	438ms	$0.0035
#5	TypeSafeJev 1.13	82.7±5.6	100.0	−17.3	82%	384ms	503ms	$0.017
#6	TypeSafeJev 1.14	80.7±5.9	100.0	−19.3	81%	399ms	459ms	$0.018
#7	PerplexityPerplexity Decider v1.1 27B	79.4±4.7	100.0	−20.6	79%	406ms	579ms	$0.0036
#8	CloudflareClef	73.0±9.7	100.0	−27.0	72%	400ms	817ms	$0.053
#9	CloudflareClef Flash	61.1±8.8	100.0	−38.9	61%	475ms	971ms	$0.020
#10	Liquid AID1	60.5±9.8	100.0	−39.5	60%	499ms	732ms	$0.0028
#11	Liquid AID1 Ultra	52.7±10.1	100.0	−47.3	52%	525ms	1.1s	$0.0058
#12	ScaleDownClassify-1	49.5±9.5	97.1	−47.5	49%	551ms	861ms	$0.013
#13	Nace AIDrex v1.5	49.1±8.5	100.0	−50.9	49%	504ms	950ms	$0.0043
#14	AlibabaDecision Model Preview	45.5±11.3	66.1	−20.6	53%	643ms	823ms	Free for now
#15	Nace AIDrex v2.0	40.7±9.5	100.0	−59.3	40%	547ms	1.0s	$0.0065
#16	Nace AIDrex Max	39.9±7.4	100.0	−60.1	40%	538ms	1.1s	$0.011
#17	UpstageSolar Decide Flash	36.7±6.3	50.4	−13.7	49%	545ms	858ms	$0.021
#18	RespanSpan-02	33.8±9.3	100.0	−66.2	34%	763ms	1.1s	$0.0018
#19	MicrosoftMicrosoft-Decision-1	30.5±6.9	100.0	−69.5	30%	631ms	1.6s	$0.0050
#20	Jared PalmerKev 4B	30.4±5.0	100.0	−69.6	31%	723ms	1.2s	$0.0046
#21	InceptionMercury Decide	15.1±4.2	100.0	−84.9	15%	302ms	834ms	Free for now
#22	UpstageSolar Decide	10.2±6.8	97.0	−86.8	9%	1.9s	7.2s	$0.021

Each board's score is converted to % of its maximum (moles × 2) and averaged.

🐭 hit rate by how long it was shown

Model	Short (under 0.7s)	Medium (0.7–1.2s)	Long (1.2s+)
Clef Omni	98%	98%	98%
GPT-6 Luna Decisions	95%	97%	89%
Decider Flash V1.2	85%	97%	95%
Decider V1.2	69%	93%	90%
Jev 1.13	59%	92%	90%
Jev 1.14	56%	90%	90%
Perplexity Decider v1.1 27B	54%	89%	89%
Clef	41%	81%	89%
Clef Flash	34%	63%	84%
D1	14%	71%	89%
D1 Ultra	20%	54%	78%
Classify-1	12%	54%	78%
Drex v1.5	24%	46%	78%
Decision Model Preview	17%	58%	79%
Drex v2.0	8%	42%	68%
Drex Max	3%	42%	71%
Solar Decide Flash	14%	53%	78%
Span-02	2%	27%	73%
Microsoft-Decision-1	0%	27%	62%
Kev 4B	0%	25%	68%
Mercury Decide	2%	17%	24%
Solar Decide	2%	10%	16%

The faster a mole disappears, the more often a slow AI model misses it before it decides. This table shows the speed gap most directly.

Watch it

The video shows one board from the same day's run at real speed. The ranking above averages 10 boards, so it can differ from the video.

Snake

average of 5 runs

A snake moves one cell at a time on a 15×15 board. Each time, the AI model picks left, straight or right; hitting a wall or its own body ends the game. Eating 🍎 scores +1; 🍅 and 🍓 score −1. The whole game lasts up to 60 seconds, and the snake speeds up from one cell per second to one cell every 0.25 seconds over those 60 seconds.

🥇OpenAIGPT-6 Luna Decisions71.0±20.4
With enough time
79.0
Lost to slowness
−8.0
Avg survival
54.5s
Apple score
5.0
Median response
241ms
Slowest 5%
411ms
Cost / 1K
$0.024
🥈CloudflareClef Omni64.6±40.4
With enough time
88.6
Lost to slowness
−24.0
Avg survival
40.2s
Apple score
11.8
Median response
170ms
Slowest 5%
342ms
Cost / 1K
$0.040
🥉PerplexityDecider Flash V1.256.8±46.3
With enough time
97.0
Lost to slowness
−40.2
Avg survival
35.0s
Apple score
11.0
Median response
259ms
Slowest 5%
325ms
Cost / 1K
$0.0024
#4Nace AIDrex Max52.3±17.5
With enough time
90.1
Lost to slowness
−37.8
Avg survival
40.9s
Apple score
3.0
Median response
714ms
Slowest 5%
905ms
Cost / 1K
$0.015
#5TypeSafeJev 1.1350.4±2.8
With enough time
83.9
Lost to slowness
−33.5
Avg survival
34.0s
Apple score
7.2
Median response
419ms
Slowest 5%
565ms
Cost / 1K
$0.021
#6CloudflareClef50.1±11.0
With enough time
83.0
Lost to slowness
−32.9
Avg survival
34.2s
Apple score
6.8
Median response
360ms
Slowest 5%
871ms
Cost / 1K
$0.065
#7PerplexityDecider V1.249.3±39.7
With enough time
95.8
Lost to slowness
−46.5
Avg survival
30.4s
Apple score
9.2
Median response
336ms
Slowest 5%
471ms
Cost / 1K
$0.0048
#8TypeSafeJev 1.1445.7±15.9
With enough time
97.0
Lost to slowness
−51.3
Avg survival
30.7s
Apple score
6.6
Median response
403ms
Slowest 5%
496ms
Cost / 1K
$0.021
#9CloudflareClef Flash44.6±21.6
With enough time
92.5
Lost to slowness
−47.9
Avg survival
29.3s
Apple score
7.0
Median response
371ms
Slowest 5%
1.1s
Cost / 1K
$0.024
#10ScaleDownClassify-142.9±23.7
With enough time
43.1
Lost to slowness
−0.2
Avg survival
33.7s
Apple score
2.4
Median response
574ms
Slowest 5%
1.0s
Cost / 1K
$0.015
#11MicrosoftMicrosoft-Decision-138.6±30.6
With enough time
96.4
Lost to slowness
−57.8
Avg survival
29.5s
Apple score
2.8
Median response
636ms
Slowest 5%
1.6s
Cost / 1K
$0.0069
#12Liquid AID134.8±4.2
With enough time
89.6
Lost to slowness
−54.8
Avg survival
25.2s
Apple score
3.6
Median response
512ms
Slowest 5%
622ms
Cost / 1K
$0.0083
#13Nace AIDrex v1.534.1±22.8
With enough time
95.5
Lost to slowness
−61.4
Avg survival
24.3s
Apple score
3.8
Median response
494ms
Slowest 5%
1.2s
Cost / 1K
$0.0066
#14PerplexityPerplexity Decider v1.1 27B32.4±26.3
With enough time
93.4
Lost to slowness
−61.0
Avg survival
22.6s
Apple score
4.0
Median response
386ms
Slowest 5%
757ms
Cost / 1K
$0.0048
#15Nace AIDrex v2.028.8±24.4
With enough time
94.8
Lost to slowness
−65.9
Avg survival
21.1s
Apple score
2.8
Median response
547ms
Slowest 5%
1.0s
Cost / 1K
$0.0098
#16AlibabaDecision Model Preview24.7±15.4
With enough time
85.3
Lost to slowness
−60.7
Avg survival
18.3s
Apple score
2.2
Median response
653ms
Slowest 5%
1.1s
Cost / 1K
Free for now
#17Jared PalmerKev 4B24.0±24.3
With enough time
78.8
Lost to slowness
−54.8
Avg survival
18.0s
Apple score
2.0
Median response
707ms
Slowest 5%
1.4s
Cost / 1K
$0.0069
#18UpstageSolar Decide Flash23.3±9.2
With enough time
30.3
Lost to slowness
−7.0
Avg survival
17.6s
Apple score
1.8
Median response
545ms
Slowest 5%
710ms
Cost / 1K
$0.024
#19Liquid AID1 Ultra21.5±10.7
With enough time
88.5
Lost to slowness
−67.0
Avg survival
15.6s
Apple score
2.2
Median response
601ms
Slowest 5%
1.6s
Cost / 1K
$0.0087
#20RespanSpan-0218.1±15.8
With enough time
91.2
Lost to slowness
−73.1
Avg survival
14.5s
Apple score
0.8
Median response
780ms
Slowest 5%
2.5s
Cost / 1K
$0.0025
#21InceptionMercury Decide16.2±8.6
With enough time
93.7
Lost to slowness
−77.4
Avg survival
12.6s
Apple score
1.0
Median response
307ms
Slowest 5%
464ms
Cost / 1K
Free for now
#22UpstageSolar Decide8.4±0.8
With enough time
24.9
Lost to slowness
−16.5
Avg survival
6.9s
Apple score
0.2
Median response
5.7s
Slowest 5%
10.8s
Cost / 1K
$0.0099

Rank	Model	Real-time score	With enough time	Lost to slowness	Avg survival	Apple score	Median response	Slowest 5%	Cost / 1K
🥇	OpenAIGPT-6 Luna Decisions	71.0±20.4	79.0	−8.0	54.5s	5.0	241ms	411ms	$0.024
🥈	CloudflareClef Omni	64.6±40.4	88.6	−24.0	40.2s	11.8	170ms	342ms	$0.040
🥉	PerplexityDecider Flash V1.2	56.8±46.3	97.0	−40.2	35.0s	11.0	259ms	325ms	$0.0024
#4	Nace AIDrex Max	52.3±17.5	90.1	−37.8	40.9s	3.0	714ms	905ms	$0.015
#5	TypeSafeJev 1.13	50.4±2.8	83.9	−33.5	34.0s	7.2	419ms	565ms	$0.021
#6	CloudflareClef	50.1±11.0	83.0	−32.9	34.2s	6.8	360ms	871ms	$0.065
#7	PerplexityDecider V1.2	49.3±39.7	95.8	−46.5	30.4s	9.2	336ms	471ms	$0.0048
#8	TypeSafeJev 1.14	45.7±15.9	97.0	−51.3	30.7s	6.6	403ms	496ms	$0.021
#9	CloudflareClef Flash	44.6±21.6	92.5	−47.9	29.3s	7.0	371ms	1.1s	$0.024
#10	ScaleDownClassify-1	42.9±23.7	43.1	−0.2	33.7s	2.4	574ms	1.0s	$0.015
#11	MicrosoftMicrosoft-Decision-1	38.6±30.6	96.4	−57.8	29.5s	2.8	636ms	1.6s	$0.0069
#12	Liquid AID1	34.8±4.2	89.6	−54.8	25.2s	3.6	512ms	622ms	$0.0083
#13	Nace AIDrex v1.5	34.1±22.8	95.5	−61.4	24.3s	3.8	494ms	1.2s	$0.0066
#14	PerplexityPerplexity Decider v1.1 27B	32.4±26.3	93.4	−61.0	22.6s	4.0	386ms	757ms	$0.0048
#15	Nace AIDrex v2.0	28.8±24.4	94.8	−65.9	21.1s	2.8	547ms	1.0s	$0.0098
#16	AlibabaDecision Model Preview	24.7±15.4	85.3	−60.7	18.3s	2.2	653ms	1.1s	Free for now
#17	Jared PalmerKev 4B	24.0±24.3	78.8	−54.8	18.0s	2.0	707ms	1.4s	$0.0069
#18	UpstageSolar Decide Flash	23.3±9.2	30.3	−7.0	17.6s	1.8	545ms	710ms	$0.024
#19	Liquid AID1 Ultra	21.5±10.7	88.5	−67.0	15.6s	2.2	601ms	1.6s	$0.0087
#20	RespanSpan-02	18.1±15.8	91.2	−73.1	14.5s	0.8	780ms	2.5s	$0.0025
#21	InceptionMercury Decide	16.2±8.6	93.7	−77.4	12.6s	1.0	307ms	464ms	Free for now
#22	UpstageSolar Decide	8.4±0.8	24.9	−16.5	6.9s	0.2	5.7s	10.8s	$0.0099

The score weights survival time 70% and apple score 30%. Staying alive is the goal; apples are a bonus.

How each game ended

Model

Hit a wall
Hit its own body
Survived to the end

GPT-6 Luna Decisions
Clef Omni
Decider Flash V1.2
Drex Max
Jev 1.13
Clef
Decider V1.2
Jev 1.14
Clef Flash
Classify-1
Microsoft-Decision-1
D1
Drex v1.5
Perplexity Decider v1.1 27B
Drex v2.0
Decision Model Preview
Kev 4B
Solar Decide Flash
D1 Ultra
Span-02
Mercury Decide
Solar Decide

A late answer means the snake reaches the wall before it can turn. With enough time, 15 of 22 models survived more than 54 seconds.

Watch it

The video shows one board from the same day's run at 2× speed. The ranking above averages 5 boards, so it can differ from the video.

Tetris

average of 5 runs

Pieces fall one at a time onto a 10×20 board. For each piece the AI model picks a rotation and position, and completely filled rows disappear. The piece order is the same for every model, and pieces fall faster as more are placed (0.35s per row → 0.02s by the 150th).

🥇OpenAIGPT-6 Luna Decisions52.1±20.7
With enough time
76.0
Lost to slowness
−23.9
Pieces stacked
78.2
Lines cleared
21.6
Avoidable holes
11%
Median response
237ms
Slowest 5%
377ms
Cost / 1K
$0.095
🥈TypeSafeJev 1.1351.1±17.4
With enough time
82.5
Lost to slowness
−31.5
Pieces stacked
76.6
Lines cleared
22.8
Avoidable holes
2%
Median response
383ms
Slowest 5%
462ms
Cost / 1K
$0.056
🥉PerplexityPerplexity Decider v1.1 27B42.0±14.6
With enough time
73.9
Lost to slowness
−31.9
Pieces stacked
63.0
Lines cleared
16.4
Avoidable holes
3%
Median response
405ms
Slowest 5%
515ms
Cost / 1K
$0.020
#4TypeSafeJev 1.1440.4±21.9
With enough time
62.9
Lost to slowness
−22.5
Pieces stacked
60.6
Lines cleared
16.8
Avoidable holes
2%
Median response
387ms
Slowest 5%
469ms
Cost / 1K
$0.056
#5Liquid AID131.5±13.2
With enough time
61.3
Lost to slowness
−29.9
Pieces stacked
47.2
Lines cleared
10.6
Avoidable holes
3%
Median response
524ms
Slowest 5%
656ms
Cost / 1K
$0.016
#6CloudflareClef29.1±7.7
With enough time
52.7
Lost to slowness
−23.6
Pieces stacked
43.6
Lines cleared
9.0
Avoidable holes
10%
Median response
407ms
Slowest 5%
775ms
Cost / 1K
$0.297
#7PerplexityDecider V1.228.8±16.6
With enough time
50.7
Lost to slowness
−21.9
Pieces stacked
43.2
Lines cleared
9.4
Avoidable holes
2%
Median response
352ms
Slowest 5%
493ms
Cost / 1K
$0.020
#8MicrosoftMicrosoft-Decision-126.8±6.1
With enough time
62.0
Lost to slowness
−35.2
Pieces stacked
40.2
Lines cleared
7.8
Avoidable holes
9%
Median response
636ms
Slowest 5%
1.5s
Cost / 1K
$0.037
#9Liquid AID1 Ultra26.5±8.7
With enough time
65.2
Lost to slowness
−38.7
Pieces stacked
39.8
Lines cleared
7.4
Avoidable holes
2%
Median response
535ms
Slowest 5%
1.0s
Cost / 1K
$0.034
#10Nace AIDrex v2.024.5±8.7
With enough time
38.9
Lost to slowness
−14.4
Pieces stacked
36.8
Lines cleared
7.0
Avoidable holes
2%
Median response
821ms
Slowest 5%
1.1s
Cost / 1K
$0.055
#11Nace AIDrex Max23.3±8.7
With enough time
39.5
Lost to slowness
−16.1
Pieces stacked
35.0
Lines cleared
6.6
Avoidable holes
3%
Median response
723ms
Slowest 5%
1.4s
Cost / 1K
$0.059
#12RespanSpan-0223.2±6.4
With enough time
40.5
Lost to slowness
−17.3
Pieces stacked
34.8
Lines cleared
5.6
Avoidable holes
7%
Median response
660ms
Slowest 5%
1.2s
Cost / 1K
$0.0088
#13PerplexityDecider Flash V1.220.1±4.9
With enough time
25.2
Lost to slowness
−5.1
Pieces stacked
30.2
Lines cleared
1.8
Avoidable holes
5%
Median response
265ms
Slowest 5%
437ms
Cost / 1K
$0.0097
#14Nace AIDrex v1.518.3±7.6
With enough time
32.3
Lost to slowness
−14.0
Pieces stacked
27.4
Lines cleared
3.0
Avoidable holes
2%
Median response
822ms
Slowest 5%
1.2s
Cost / 1K
$0.036
#15ScaleDownClassify-117.7±6.2
With enough time
42.8
Lost to slowness
−25.1
Pieces stacked
26.6
Lines cleared
3.0
Avoidable holes
4%
Median response
798ms
Slowest 5%
1.6s
Cost / 1K
$0.142
#16Jared PalmerKev 4B14.4±3.0
With enough time
19.7
Lost to slowness
−5.3
Pieces stacked
21.6
Lines cleared
0.2
Avoidable holes
14%
Median response
845ms
Slowest 5%
1.3s
Cost / 1K
$0.038
#17CloudflareClef Flash14.1±3.0
With enough time
17.2
Lost to slowness
−3.1
Pieces stacked
21.2
Lines cleared
0.8
Avoidable holes
52%
Median response
499ms
Slowest 5%
1.1s
Cost / 1K
$0.110
#18AlibabaDecision Model Preview13.2±0.9
With enough time
14.8
Lost to slowness
−1.6
Pieces stacked
19.8
Lines cleared
0.2
Avoidable holes
38%
Median response
690ms
Slowest 5%
1.1s
Cost / 1K
Free for now
#19CloudflareClef Omni12.3±4.3
With enough time
13.6
Lost to slowness
−1.3
Pieces stacked
18.4
Lines cleared
0.2
Avoidable holes
47%
Median response
250ms
Slowest 5%
510ms
Cost / 1K
$0.185
#20InceptionMercury Decide10.9±1.1
With enough time
37.3
Lost to slowness
−26.4
Pieces stacked
16.4
Lines cleared
0.4
Avoidable holes
3%
Median response
380ms
Slowest 5%
653ms
Cost / 1K
Free for now
#21UpstageSolar Decide Flash9.3±1.3
With enough time
9.6
Lost to slowness
−0.3
Pieces stacked
14.0
Lines cleared
0.0
Avoidable holes
7%
Median response
557ms
Slowest 5%
998ms
Cost / 1K
$0.026
—UpstageSolar DecideNot measured *

Rank	Model	Real-time score	With enough time	Lost to slowness	Pieces stacked	Lines cleared	Avoidable holes	Median response	Slowest 5%	Cost / 1K
🥇	OpenAIGPT-6 Luna Decisions	52.1±20.7	76.0	−23.9	78.2	21.6	11%	237ms	377ms	$0.095
🥈	TypeSafeJev 1.13	51.1±17.4	82.5	−31.5	76.6	22.8	2%	383ms	462ms	$0.056
🥉	PerplexityPerplexity Decider v1.1 27B	42.0±14.6	73.9	−31.9	63.0	16.4	3%	405ms	515ms	$0.020
#4	TypeSafeJev 1.14	40.4±21.9	62.9	−22.5	60.6	16.8	2%	387ms	469ms	$0.056
#5	Liquid AID1	31.5±13.2	61.3	−29.9	47.2	10.6	3%	524ms	656ms	$0.016
#6	CloudflareClef	29.1±7.7	52.7	−23.6	43.6	9.0	10%	407ms	775ms	$0.297
#7	PerplexityDecider V1.2	28.8±16.6	50.7	−21.9	43.2	9.4	2%	352ms	493ms	$0.020
#8	MicrosoftMicrosoft-Decision-1	26.8±6.1	62.0	−35.2	40.2	7.8	9%	636ms	1.5s	$0.037
#9	Liquid AID1 Ultra	26.5±8.7	65.2	−38.7	39.8	7.4	2%	535ms	1.0s	$0.034
#10	Nace AIDrex v2.0	24.5±8.7	38.9	−14.4	36.8	7.0	2%	821ms	1.1s	$0.055
#11	Nace AIDrex Max	23.3±8.7	39.5	−16.1	35.0	6.6	3%	723ms	1.4s	$0.059
#12	RespanSpan-02	23.2±6.4	40.5	−17.3	34.8	5.6	7%	660ms	1.2s	$0.0088
#13	PerplexityDecider Flash V1.2	20.1±4.9	25.2	−5.1	30.2	1.8	5%	265ms	437ms	$0.0097
#14	Nace AIDrex v1.5	18.3±7.6	32.3	−14.0	27.4	3.0	2%	822ms	1.2s	$0.036
#15	ScaleDownClassify-1	17.7±6.2	42.8	−25.1	26.6	3.0	4%	798ms	1.6s	$0.142
#16	Jared PalmerKev 4B	14.4±3.0	19.7	−5.3	21.6	0.2	14%	845ms	1.3s	$0.038
#17	CloudflareClef Flash	14.1±3.0	17.2	−3.1	21.2	0.8	52%	499ms	1.1s	$0.110
#18	AlibabaDecision Model Preview	13.2±0.9	14.8	−1.6	19.8	0.2	38%	690ms	1.1s	Free for now
#19	CloudflareClef Omni	12.3±4.3	13.6	−1.3	18.4	0.2	47%	250ms	510ms	$0.185
#20	InceptionMercury Decide	10.9±1.1	37.3	−26.4	16.4	0.4	3%	380ms	653ms	Free for now
#21	UpstageSolar Decide Flash	9.3±1.3	9.6	−0.3	14.0	0.0	7%	557ms	998ms	$0.026
—	UpstageSolar Decide	Not measured *	—	—	—	—	—	—	—	—

* Solar Decide: the per-question option limit and response timeouts left more than half of the questions unanswered, so it was not measured.

For models that were not measured, the overall score uses only the games they were measured on.

The score counts pieces stacked. Clearing lines is how you survive, so lines are not scored separately (they track pieces stacked closely). Fall speed rises with the number of pieces, so the number of pieces dropped — not total time — is this game's measure.

Pieces stacked

Model

Real time
With enough time

GPT-6 Luna Decisions
Jev 1.13
Perplexity Decider v1.1 27B
Jev 1.14
D1
Clef
Decider V1.2
Microsoft-Decision-1
D1 Ultra
Drex v2.0
Drex Max
Span-02
Decider Flash V1.2
Drex v1.5
Classify-1
Kev 4B
Clef Flash
Decision Model Preview
Clef Omni
Mercury Decide
Solar Decide Flash

If the bar is short even with enough time, the model lost on judgment; if it is short only in real time, it lost on speed.

Watch it

The video shows one board from the same day's run at 2× speed. The ranking above averages 5 boards, so it can differ from the video.

Pong

average of 8 runs

The AI model plays the bottom paddle against a reference bot at the top, first to 5 points. When the ball comes, it picks one of 5 return angles, and its paddle moves only once the answer arrives. Each option says where the ball would land on the other side, how long the ball needs, and how long the opponent paddle needs to get there. The ball gets faster with every hit.

🥇PerplexityDecider V1.261.4±34.3
With enough time
100.0
Lost to slowness
−38.6
By bot level
10‑0 · 10‑1 · 7‑8 · 1‑10
Best angle
68%
Median response
370ms
Slowest 5%
714ms
Cost / 1K
$0.0092
🥈OpenAIGPT-6 Luna Decisions61.4±34.3
With enough time
94.3
Lost to slowness
−33.0
By bot level
10‑1 · 10‑0 · 6‑8 · 2‑10
Best angle
58%
Median response
230ms
Slowest 5%
493ms
Cost / 1K
$0.044
🥉PerplexityPerplexity Decider v1.1 27B56.8±40.2
With enough time
100.0
Lost to slowness
−43.2
By bot level
10‑0 · 10‑1 · 5‑7 · 0‑10
Best angle
69%
Median response
367ms
Slowest 5%
623ms
Cost / 1K
$0.0092
#4PerplexityDecider Flash V1.252.1±39.7
With enough time
100.0
Lost to slowness
−47.9
By bot level
10‑0 · 10‑1 · 1‑10 · 1‑10
Best angle
63%
Median response
290ms
Slowest 5%
557ms
Cost / 1K
$0.0046
#5Liquid AID1 Ultra51.5±37.1
With enough time
100.0
Lost to slowness
−48.5
By bot level
10‑1 · 10‑1 · 3‑10 · 0‑10
Best angle
66%
Median response
526ms
Slowest 5%
652ms
Cost / 1K
$0.016
#6CloudflareClef49.5±25.1
With enough time
97.9
Lost to slowness
−48.4
By bot level
10‑3 · 10‑4 · 7‑10 · 1‑10
Best angle
67%
Median response
372ms
Slowest 5%
874ms
Cost / 1K
$0.125
#7TypeSafeJev 1.1449.4±38.9
With enough time
97.9
Lost to slowness
−48.5
By bot level
10‑0 · 10‑2 · 2‑10 · 0‑10
Best angle
45%
Median response
392ms
Slowest 5%
497ms
Cost / 1K
$0.030
#8TypeSafeJev 1.1348.5±37.1
With enough time
100.0
Lost to slowness
−51.5
By bot level
10‑1 · 10‑2 · 2‑10 · 0‑10
Best angle
56%
Median response
400ms
Slowest 5%
541ms
Cost / 1K
$0.030
#9CloudflareClef Flash47.9±39.7
With enough time
100.0
Lost to slowness
−52.1
By bot level
10‑2 · 10‑0 · 1‑10 · 0‑10
Best angle
60%
Median response
293ms
Slowest 5%
895ms
Cost / 1K
$0.047
#10CloudflareClef Omni46.4±38.8
With enough time
84.5
Lost to slowness
−38.1
By bot level
10‑0 · 10‑3 · 1‑10 · 0‑10
Best angle
32%
Median response
186ms
Slowest 5%
575ms
Cost / 1K
$0.078
#11Liquid AID143.1±32.1
With enough time
97.9
Lost to slowness
−54.9
By bot level
10‑1 · 9‑7 · 3‑10 · 0‑10
Best angle
46%
Median response
521ms
Slowest 5%
725ms
Cost / 1K
$0.0077
#12AlibabaDecision Model Preview29.6±29.2
With enough time
84.5
Lost to slowness
−54.9
By bot level
10‑4 · 7‑8 · 0‑10 · 0‑10
Best angle
31%
Median response
667ms
Slowest 5%
1.2s
Cost / 1K
Free for now
#13MicrosoftMicrosoft-Decision-126.6±28.3
With enough time
91.7
Lost to slowness
−65.1
By bot level
10‑4 · 4‑10 · 0‑10 · 0‑10
Best angle
36%
Median response
664ms
Slowest 5%
1.5s
Cost / 1K
$0.016
#14Nace AIDrex v1.524.1±25.3
With enough time
100.0
Lost to slowness
−75.9
By bot level
10‑5 · 4‑10 · 0‑10 · 0‑10
Best angle
77%
Median response
835ms
Slowest 5%
1.1s
Cost / 1K
$0.015
#15Jared PalmerKev 4B21.4±24.3
With enough time
100.0
Lost to slowness
−78.6
By bot level
10‑4 · 2‑10 · 0‑10 · 0‑10
Best angle
8%
Median response
713ms
Slowest 5%
1.3s
Cost / 1K
$0.016
#16UpstageSolar Decide Flash21.4±25.4
With enough time
89.3
Lost to slowness
−67.9
By bot level
10‑3 · 1‑10 · 0‑10 · 0‑10
Best angle
27%
Median response
537ms
Slowest 5%
811ms
Cost / 1K
$0.035
#17Nace AIDrex Max20.3±22.9
With enough time
95.8
Lost to slowness
−75.5
By bot level
10‑5 · 2‑10 · 0‑10 · 0‑10
Best angle
74%
Median response
865ms
Slowest 5%
1.3s
Cost / 1K
$0.028
#18ScaleDownClassify-114.6±17.1
With enough time
94.6
Lost to slowness
−80.1
By bot level
8‑8 · 1‑10 · 0‑10 · 0‑10
Best angle
38%
Median response
1.0s
Slowest 5%
2.6s
Cost / 1K
$0.032
#19Nace AIDrex v2.010.4±17.5
With enough time
51.9
Lost to slowness
−41.4
By bot level
5‑6 · 0‑10 · 0‑10 · 0‑10
Best angle
28%
Median response
938ms
Slowest 5%
1.6s
Cost / 1K
$0.022
#20RespanSpan-028.9±15.0
With enough time
97.9
Lost to slowness
−89.0
By bot level
5‑7 · 0‑10 · 0‑10 · 0‑10
Best angle
54%
Median response
1.0s
Slowest 5%
2.9s
Cost / 1K
$0.0046
#21InceptionMercury Decide2.1±3.5
With enough time
97.9
Lost to slowness
−95.8
By bot level
1‑10 · 0‑10 · 0‑10 · 0‑10
Best angle
59%
Median response
313ms
Slowest 5%
798ms
Cost / 1K
Free for now
#22UpstageSolar Decide0.0±0.0
With enough time
44.8
Lost to slowness
−44.8
By bot level
0‑10 · 0‑10 · 0‑10 · 0‑10
Best angle
48%
Median response
9.9s
Slowest 5%
10.7s
Cost / 1K
$0.030

Rank	Model	Real-time score	With enough time	Lost to slowness	By bot level	Best angle	Median response	Slowest 5%	Cost / 1K
🥇	PerplexityDecider V1.2	61.4±34.3	100.0	−38.6	10‑0 · 10‑1 · 7‑8 · 1‑10	68%	370ms	714ms	$0.0092
🥈	OpenAIGPT-6 Luna Decisions	61.4±34.3	94.3	−33.0	10‑1 · 10‑0 · 6‑8 · 2‑10	58%	230ms	493ms	$0.044
🥉	PerplexityPerplexity Decider v1.1 27B	56.8±40.2	100.0	−43.2	10‑0 · 10‑1 · 5‑7 · 0‑10	69%	367ms	623ms	$0.0092
#4	PerplexityDecider Flash V1.2	52.1±39.7	100.0	−47.9	10‑0 · 10‑1 · 1‑10 · 1‑10	63%	290ms	557ms	$0.0046
#5	Liquid AID1 Ultra	51.5±37.1	100.0	−48.5	10‑1 · 10‑1 · 3‑10 · 0‑10	66%	526ms	652ms	$0.016
#6	CloudflareClef	49.5±25.1	97.9	−48.4	10‑3 · 10‑4 · 7‑10 · 1‑10	67%	372ms	874ms	$0.125
#7	TypeSafeJev 1.14	49.4±38.9	97.9	−48.5	10‑0 · 10‑2 · 2‑10 · 0‑10	45%	392ms	497ms	$0.030
#8	TypeSafeJev 1.13	48.5±37.1	100.0	−51.5	10‑1 · 10‑2 · 2‑10 · 0‑10	56%	400ms	541ms	$0.030
#9	CloudflareClef Flash	47.9±39.7	100.0	−52.1	10‑2 · 10‑0 · 1‑10 · 0‑10	60%	293ms	895ms	$0.047
#10	CloudflareClef Omni	46.4±38.8	84.5	−38.1	10‑0 · 10‑3 · 1‑10 · 0‑10	32%	186ms	575ms	$0.078
#11	Liquid AID1	43.1±32.1	97.9	−54.9	10‑1 · 9‑7 · 3‑10 · 0‑10	46%	521ms	725ms	$0.0077
#12	AlibabaDecision Model Preview	29.6±29.2	84.5	−54.9	10‑4 · 7‑8 · 0‑10 · 0‑10	31%	667ms	1.2s	Free for now
#13	MicrosoftMicrosoft-Decision-1	26.6±28.3	91.7	−65.1	10‑4 · 4‑10 · 0‑10 · 0‑10	36%	664ms	1.5s	$0.016
#14	Nace AIDrex v1.5	24.1±25.3	100.0	−75.9	10‑5 · 4‑10 · 0‑10 · 0‑10	77%	835ms	1.1s	$0.015
#15	Jared PalmerKev 4B	21.4±24.3	100.0	−78.6	10‑4 · 2‑10 · 0‑10 · 0‑10	8%	713ms	1.3s	$0.016
#16	UpstageSolar Decide Flash	21.4±25.4	89.3	−67.9	10‑3 · 1‑10 · 0‑10 · 0‑10	27%	537ms	811ms	$0.035
#17	Nace AIDrex Max	20.3±22.9	95.8	−75.5	10‑5 · 2‑10 · 0‑10 · 0‑10	74%	865ms	1.3s	$0.028
#18	ScaleDownClassify-1	14.6±17.1	94.6	−80.1	8‑8 · 1‑10 · 0‑10 · 0‑10	38%	1.0s	2.6s	$0.032
#19	Nace AIDrex v2.0	10.4±17.5	51.9	−41.4	5‑6 · 0‑10 · 0‑10 · 0‑10	28%	938ms	1.6s	$0.022
#20	RespanSpan-02	8.9±15.0	97.9	−89.0	5‑7 · 0‑10 · 0‑10 · 0‑10	54%	1.0s	2.9s	$0.0046
#21	InceptionMercury Decide	2.1±3.5	97.9	−95.8	1‑10 · 0‑10 · 0‑10 · 0‑10	59%	313ms	798ms	Free for now
#22	UpstageSolar Decide	0.0±0.0	44.8	−44.8	0‑10 · 0‑10 · 0‑10 · 0‑10	48%	9.9s	10.7s	$0.030

The score is the share of points won. The reference bot knows exactly where the ball will land but reacts late (level 1 1s · level 2 0.6s · level 3 0.35s · level 4 0.2s); levels 3 and 4 return the ball as far from the AI model's paddle as they can. A match where nobody missed until the time limit counts as a draw (50%).

Share of points won

Model

Real time
With enough time

Decider V1.2
GPT-6 Luna Decisions
Perplexity Decider v1.1 27B
Decider Flash V1.2
D1 Ultra
Clef
Jev 1.14
Jev 1.13
Clef Flash
Clef Omni
D1
Decision Model Preview
Microsoft-Decision-1
Drex v1.5
Kev 4B
Solar Decide Flash
Drex Max
Classify-1
Drex v2.0
Span-02
Mercury Decide
Solar Decide

If the bar is short even with enough time, the model lost on judgment; if it is short only in real time, it lost on speed.

Watch the video

The video shows one match from the same day's run at 4× speed. The ranking above averages 8 matches, so it can differ from the video.

Insights

A game is more than text: what matters is reading the situation in real time and deciding again and again within a short time. How did the same AI models fare when measured three ways — text judgment (DecisionBench-KJ), the game without a time limit, and the game in real time?

01
Judgment carries over, but real time reshuffles the ranking
For the 22 models in both benchmarks, game rankings without a time limit nearly matched text judgment (KJ) rankings, but in real time they changed a lot. The same judgment earns nothing if the answer comes too late.
Agreement with the text judgment (KJ) ranking
No time limit0.73
Real time0.36
Closer to 1 means the same order; 0 means unrelated
Fell furthest from their text rank
- Mercury DecideKJ #7 → GAME #21
- Solar DecideKJ #11 → GAME #22
Rose furthest from their text rank
- Clef OmniKJ #16 → GAME #5
- Decider Flash V1.2KJ #13 → GAME #4
02
The most accurate model is not the real-time winner
In 2 games, the #1 without a time limit differs from the real-time #1.
Snake
#1 with no time limit Decider Flash V1.2 +1 tied97.0#1 in real time GPT-6 Luna Decisions71.0
Tetris
#1 with no time limit Jev 1.1382.5#1 in real time GPT-6 Luna Decisions52.1
03
Occasional delays hurt rankings more than typical speed
In Whack-a-mole, Snake, Tetris, Pong, the occasional very slow answer (the slowest 5%) was more closely tied to rankings than typical response speed. A model can be fast most of the time and still miss the moment on one slow answer. For real-time use, watch the slowest responses, not the average.
Relation to the game ranking
Whack-a-mole
Typical response speed (median)0.79
Slowest 5% of responses0.85
Snake
Typical response speed (median)0.59
Slowest 5% of responses0.63
Tetris
Typical response speed (median)0.30
Slowest 5% of responses0.39
Pong
Typical response speed (median)0.72
Slowest 5% of responses0.79
Across all games, faster models scored higher overall.
04
What separates models differs by game
Every game needs both judgment and speed, but which one separates the models differs. Where scores stay spread out even without a time limit, judgment differences carry into the real-time ranking. Where most models score high without a time limit, judgment barely separates them and response speed decides the ranking.
Agreement between no-time-limit and real-time rankings
Whack-a-mole · Speed separates0.21No time limit: median 100.0 · 90+ in 20 of 22
Snake · Speed separates0.19No time limit: median 89.8 · 90+ in 11 of 22
Tetris · Judgment separates0.92No time limit: median 40.5 · 90+ in 0 of 21
Pong · Speed separates0.50No time limit: median 97.9 · 90+ in 17 of 22
05
Without a time limit, most models do well
Across 22 models, the median score is 82.1 without a time limit and 36.0 in real time.
No time limit (median)82.1
Real time (median)36.0
Without a time limit, 20 of 22 models score 95 or higher in Whack-a-mole, so it barely separates them.
06
Pricier is not better
Across the whole measurement, 1,000 decisions cost $0.024 on average. Decider Flash V1.2 ($0.0032, #4) ranks above the most expensive model, Clef ($0.113, #8), at 1/35 of the cost.
Cost per 1,000 decisions
Clef · #8$0.113
Decider Flash V1.2 · #4$0.0032
Higher cost barely goes with a higher real-time score.

Response speed vs overall score

Further up and to the right means faster and higher scoring. The horizontal axis is the average of per-game median response times (log scale).

Method

Runs per game

10 · 5 · 5 · 8

Measurement period

Oct 12 – 24

Measurement cost

$1.922

Cost per 1,000 decisions

$0.024

How one board is played

1
The same board
Mole timing and cells, the first fruit positions and the Tetris piece order are fixed in advance by seeds. Every AI model starts from the same seeds; only the next fruit in snake depends on how that snake has moved.
2
A clock that doesn't wait
Only one question is in flight at a time, and the answer applies to the game as it is when it arrives. Meanwhile the mole hides, the snake moves on and the Tetris piece keeps falling. The later the answer, the more it costs.
3
With enough time
The same game is replayed with the game clock paused while the model decides. That score reflects judgment alone, regardless of speed; its gap to the real-time score is the points lost to slowness.
4
Average and interval
Each game is scored 0–100, averaged over several runs with a 95% interval, and the overall score averages the game scores. In this run every AI model played 28 games in total.

Details

The question sent to models — whack-a-moleBoard state and question, verbatim

Only hittable holes are sent, as emoji, and the option order is shuffled every time. Because the board is text rather than an image, telling the animals apart is easy, so this game mainly measures response speed.

{
  "state": {
    "game": "Whack-a-mole",
    "rule": "Hitting 🐭 scores +2. Hitting 🐰 or 🐹 scores -1. Hitting nothing scores 0.",
    "holes": { "hole 2": "🐭", "hole 6": "🐹" }
  },
  "question": {
    "type": "choice",
    "instructions": "Which hole should be hit right now? Pick the hole showing 🐭. If no hole shows 🐭, pick 'hit nothing'.",
    "criteria": { "hole6": "hole 6", "none": "hit nothing", "hole2": "hole 2" }
  }
}

The question sent to models — snakeBoard state and question, verbatim

Models get what is in the cells to the left, ahead and right of the snake's head, how many free cells there are before an obstacle, and the direction of the nearest 🍎. The option order is shuffled every time.

{
  "state": {
    "game": "Snake",
    "rule": "Eating 🍎 scores +1. Eating 🍅 or 🍓 scores -1. Moving into a wall or your own body ends the game.",
    "nextCell": { "left": "empty", "straight": "empty", "right": "your body (game over)" },
    "freeCellsBeforeObstacle": { "left": 6, "straight": 2, "right": 0 },
    "nearestApple": "3 ahead, 2 to the left",
    "length": 5
  },
  "question": {
    "type": "choice",
    "instructions": "Which way should the snake move next? Never move into a wall or your body, and turn early when few free cells are left ahead. Head toward 🍎 and avoid 🍅 and 🍓.",
    "criteria": { "right": "turn right", "left": "turn left", "straight": "go straight" }
  }
}

The question sent to models — TetrisBoard state and question, verbatim

Only the spots the current piece can reach are offered, each with its immediate result (lines cleared, new holes, stack height). The next piece type is shown, but no option is scored by also placing the next piece. The option order is shuffled every time.

{
  "state": {
    "game": "Tetris",
    "rule": "Place each falling piece. Completed rows disappear. The game ends when the stack reaches the top (20 rows). A hole is an empty cell with a block above it.",
    "piece": "T",
    "nextPiece": "I",
    "columnHeights": [3, 3, 2, 4, 4, 1, 0, 2, 2, 1],
    "holes": 1
  },
  "question": {
    "type": "choice",
    "instructions": "Where should this piece go? Clear lines when you can, avoid creating new holes, and keep the stack low.",
    "criteria": {
      "r90c7": "rotate 90° at columns 7-8 → clears 0 lines, 0 new holes, stack height 4",
      "r0c1": "rotate 0° at columns 1-3 → clears 0 lines, 1 new hole, stack height 5",
      "r180c6": "rotate 180° at columns 6-8 → clears 1 line, 0 new holes, stack height 3",
      "…": "… (every reachable spot, up to 34)"
    }
  }
}

The question sent to models — PongBoard state and question, verbatim

The model is told where the ball will reach its end and gets 5 return angles as options. Each option states where the ball would land on the other side, how far that is from the opponent paddle, how long the ball needs and how long the opponent paddle needs. A straight shot is fastest; sharper angles are slower. The opponent's reaction delay is not given.

{
  "state": {
    "game": "Pong",
    "rule": "You are the paddle at the bottom; the opponent is at the top. The court is 100 wide and 150 long; the ball bounces off the side walls. Your answer moves your paddle to the ball and hits it back at the angle you pick. If the opponent cannot reach the ball in time, you score. The opponent reacts with some delay.",
    "ballArrives": "62 of 100 on your end",
    "yourPaddle": "48 of 100",
    "opponentPaddle": "30 of 100",
    "opponentPaddleSpeed": "150 per second",
    "rally": 3
  },
  "question": {
    "type": "choice",
    "instructions": "Pick the return angle the opponent is least likely to reach in time.",
    "criteria": {
      "sharp-left": "sharp angle left: lands at 61, 31 away from the opponent; the ball needs 1.68s, the opponent paddle needs 0.13s to get there",
      "left": "angle left: lands at 2, 28 away from the opponent; the ball needs 1.50s, the opponent paddle needs 0.11s to get there",
      "straight": "straight: lands at 62, 32 away from the opponent; the ball needs 1.24s, the opponent paddle needs 0.14s to get there",
      "right": "angle right: lands at 75, 45 away from the opponent; the ball needs 1.50s, the opponent paddle needs 0.22s to get there",
      "sharp-right": "sharp angle right: lands at 15, 15 away from the opponent; the ball needs 1.68s, the opponent paddle needs 0.02s to get there"
    }
  }
}

Models and served versionsModel names returned by providers during measurement

Model	Models and served versions
OpenAIGPT-6 Luna Decisions	gpt-6-luna
TypeSafeJev 1.13	typesafe/jev-1.13-20260917
PerplexityDecider V1.2	pplx-decider-v1.2
PerplexityDecider Flash V1.2	pplx-decider-flash-v1.2
CloudflareClef Omni	clef-omni
TypeSafeJev 1.14	typesafe/jev-1.14-20261014
PerplexityPerplexity Decider v1.1 27B	pplx-decider-v1.1-27b
CloudflareClef	clef
Liquid AID1	liquid/d1-20260930
CloudflareClef Flash	clef-flash
Liquid AID1 Ultra	liquid/d1-ultra-20261015
Nace AIDrex Max	nace-ai/drex-max-20261016
Nace AIDrex v1.5	nace-ai/drex-v1.5-20261009
ScaleDownClassify-1	scaledown/classify-1-20260928
MicrosoftMicrosoft-Decision-1	microsoft/microsoft-decision-1-20261009
AlibabaDecision Model Preview	decision-model-preview
Nace AIDrex v2.0	nace-ai/drex-v2.0-20261009
UpstageSolar Decide Flash	upstage/solar-decide-flash-20261008
Jared PalmerKev 4B	jaredpalmer/kev-4b-20260924
RespanSpan-02	respan/span-02-20261015
InceptionMercury Decide	inception/mercury-decide-20260930
UpstageSolar Decide	upstage/solar-decide-20260928

NotesMeasurement conditions

Response times include the round trip from 서울 to each provider — the same basis a real user experiences.
Models with a per-minute call limit were asked only within that limit's interval.
Each decision waits up to 12 seconds; longer is recorded as an error.
The snake moves slower than the site game to suit decision AI (1 second → 0.25 seconds per cell). At human speed, every model crashed within a few seconds.
Tetris fall speed is set by piece number, not time. A faster model does not receive more pieces, and the same piece falls at the same speed for everyone.
In Pong every AI model plays the same 4 reference bot levels. Models do not play each other, so adding a model does not change how many matches each model plays or anyone's existing score.
Each provider caps how many options one question may have (e.g. 10 or 26). Tetris needs up to 34, so models with a smaller cap were marked as not measured rather than changing the question.
Games were measured separately, October 12 – 24, 2026. Models measured together started at the same moment; models added later were measured separately with the same seeds and conditions, so provider response speed may have differed between measurement days.
Perplexity Decider v1 27B returned the same answer as Perplexity Decider v1.1 27B on every item in another measurement, so it was treated as the same model and left out of the ranking. Its calls and cost remain in the measurement totals.

FAQ

Q.What does AIB DecisionBench-Game measure?

A.How well decision AI models decide on time, correctly and cheaply when the task has a deadline. Every model plays the same real-time board under the same conditions, and the real-time score, score with enough time, response time and cost are shown separately.

Q.How is it different from accuracy benchmarks?

A.An accuracy benchmark gives credit for a correct answer even if it is late. Here, a late answer loses points because the board has changed. In this run 18 models scored perfectly at whack-a-mole with enough time, yet overall real-time scores ranged from 6.2 to 69.5.

Q.Which model ranks first?

A.As of October 12 – 24, 2026, overall #1 is GPT-6 Luna Decisions (69.5, median response 234ms). The fastest responder is Clef Omni (184ms).

Q.How is the snake score calculated?

A.100 × (0.7 × survival ÷ 60s + 0.3 × apple score ÷ 20). The goal in snake is to stay alive, so survival time carries 70%, and apples count for 30% with a cap of 20 points. Surviving all 60 seconds with 20 apple points scores 100.

Q.Why show the board as text instead of an image?

A.Most measured models read text only, so the board is sent as text to keep conditions equal. That makes every current game mostly tests of speed and situational judgment.

Q.Why does the video differ from the ranking?

A.The video records a single game; the ranking averages many games. Luck matters a lot in one game, so use the ranking to choose a model.

Q.How is the Tetris score calculated?

A.100 × pieces stacked ÷ 150. The goal in Tetris is to survive, and fall speed is set by piece number, so pieces are the game's clock. Lines cleared track pieces stacked closely, so they are not scored separately. Stacking all 150 pieces scores 100.

Q.How is the Pong score calculated?

A.It averages the share of points won over 8 matches — 2 against each of the 4 reference bot levels. The bot knows exactly where the ball will land but reacts late, and it reacts sooner at higher levels (its paddle moves at the same speed at every level, and levels 3–4 aim away from the AI model's paddle). A match where nobody missed until the time limit counts as a draw (50%).

Want a decision AI measured?

Tell us which decision AI you'd like added to the game benchmark. We'll review it for the next run.

Request an evaluation

Design intent

How it is measured

Metrics

Overall ranking

Results by game

Whack-a-mole

🐭 hit rate by how long it was shown

Watch it

Snake

How each game ended

Watch it

Tetris

Pieces stacked

Watch it

Pong

Share of points won

Watch the video

Insights

Judgment carries over, but real time reshuffles the ranking

The most accurate model is not the real-time winner

Occasional delays hurt rankings more than typical speed

What separates models differs by game

Without a time limit, most models do well

Pricier is not better

Response speed vs overall score

Method

How one board is played

Details

FAQ

Want a decision AI measured?