L. Cordero published a July 27 account of “Second Squeeze,” a local AI benchmarking project that compared GPT-2 XL from 2019 with Qwen3-0.6B from 2025 on the same laptop after seeing a model answer locally at AMD's Advancing AI 2026 summit. The project used Lemonade, AMD's local AI server, to run GPT-2 XL with 1.5 billion parameters and Qwen3-0.6B, a 2025 model less than half that size.

Cordero said the setup took a week and hit several installation problems: the command differed from what the docs implied, Lemonade was not on PATH at first, a GPU backend download jammed twice on a locked file, and the evaluation tool required a second Python version because the installed Python was too new. The first goal was to test GPT-2 against a 2019 accuracy benchmark, but Lemonade did not return the probability data that the standard tool expected.

Cordero and Claude traced the missing probability data, reproduced the failure, and filed Lemonade issue 2822 as the author's first open source contribution. The benchmark then shifted from accuracy to speed and basic instruction following, using reproducible scripts so every number could be rerun with one command.

The first speed result put GPT-2 at 0.31 tokens per second, compared with 90 tokens per second for the newer model, implying a 297x gap. After rerunning the scripted benchmark, GPT-2 measured 8.5 tokens per second, not 0.31, because the earlier run occurred while a backend was still downloading and the model was cold from disk; the revised performance gap was about 11x, not 297x.

On a simple completion prompt, “The capital of France is,” Qwen3-0.6B answered Paris and showed brief reasoning, while GPT-2 XL produced random import statements. Second Squeeze went live with a dashboard, an MIT-licensed repo, tests, and the Lemonade issue, and Cordero said the next step is running the project on an actual AMD GPU.

L. Cordero published a July 27 account of “Second Squeeze,” a local AI benchmarking project that compared GPT-2 XL from 2019 with Qwen3-0.6B from 2025 on the same laptop after seeing a model answer locally at AMD's Advancing AI 2026 summit. The project used Lemonade, AMD's local AI server, to run GPT-2 XL with 1.5 billion parameters and Qwen3-0.6B, a 2025 model less than half that size.

Cordero said the setup took a week and hit several installation problems: the command differed from what the docs implied, Lemonade was not on PATH at first, a GPU backend download jammed twice on a locked file, and the evaluation tool required a second Python version because the installed Python was too new. The first goal was to test GPT-2 against a 2019 accuracy benchmark, but Lemonade did not return the probability data that the standard tool expected.

Cordero and Claude traced the missing probability data, reproduced the failure, and filed Lemonade issue 2822 as the author's first open source contribution. The benchmark then shifted from accuracy to speed and basic instruction following, using reproducible scripts so every number could be rerun with one command.

The first speed result put GPT-2 at 0.31 tokens per second, compared with 90 tokens per second for the newer model, implying a 297x gap. After rerunning the scripted benchmark, GPT-2 measured 8.5 tokens per second, not 0.31, because the earlier run occurred while a backend was still downloading and the model was cold from disk; the revised performance gap was about 11x, not 297x.

On a simple completion prompt, “The capital of France is,” Qwen3-0.6B answered Paris and showed brief reasoning, while GPT-2 XL produced random import statements. Second Squeeze went live with a dashboard, an MIT-licensed repo, tests, and the Lemonade issue, and Cordero said the next step is running the project on an actual AMD GPU.