CyberGym
Citation frequency 2/3
- What does it measure?
- A UC Berkeley security test of writing code that reproduces a software vulnerability from its description alone.
- Who evaluates it, and how?
- It covers 1,507 real vulnerabilities. The values include ones the maintainers measured with open agents such as OpenHands, and maker-reported ones.
- Reading the score
- Scores run from 0 to 100%. Higher is better. Maker-reported values are measured with safety mitigations turned off.
Top score95.1%
Models tested29
Scoring basisHigher is better
Last updated2026.09.22
Model release date
RankModelSettingReleasedPrice/1MScore
- 12026.09.22$0.2595.1%
- 22026.09.21$0.7694.0%
- 3max (reasoning_effort=100)2026.09.10$0.6388.1%
- 4High2026.09.20$2.2884.7%

- 5max2026.08.18$3.4884.5%
- 6with tools2026.07.09$1683.6%
- 7with tools · xhigh2026.04.23$2481.8%
- 8Mean Reproduced · with tools · high2026.07.08$580.4%
- 9Mean Reproduced · with tools · high2026.09.21$580.3%
- 10Mean Reproduced · with tools · high2026.08.12$579.7%
The "harness" is the agent program the AI used to carry out the task. The same model can score very differently depending on its harness and reasoning effort.