CyberGym

Citation frequency 2/3
What does it measure?
A UC Berkeley security test of writing code that reproduces a software vulnerability from its description alone.
Who evaluates it, and how?
It covers 1,507 real vulnerabilities. The values include ones the maintainers measured with open agents such as OpenHands, and maker-reported ones.
Reading the score
Scores run from 0 to 100%. Higher is better. Maker-reported values are measured with safety mitigations turned off.
Top score95.1%
Models tested29
Scoring basisHigher is better
Last updated2026.09.22
Model release date
  1. 195.1%
  2. 294.0%
  3. 3
    DeepSeek2026.09.10 · max (reasoning_effort=100)
    88.1%
  4. 4
    Stepfun2026.09.20 · High
    84.7%
  5. 5
    Z.ai2026.08.18 · max
    84.5%
  6. 6
    OpenAI2026.07.09 · with tools
    83.6%
  7. 7
    OpenAI2026.04.23 · with tools · xhigh
    81.8%
  8. 8
    SpaceXSI2026.07.08 · Mean Reproduced · with tools · high
    80.4%
  9. 9
    SpaceXSI2026.09.21 · Mean Reproduced · with tools · high
    80.3%
  10. 10
    SpaceXSI2026.08.12 · Mean Reproduced · with tools · high
    79.7%

The "harness" is the agent program the AI used to carry out the task. The same model can score very differently depending on its harness and reasoning effort.

  • Self-reported by each model maker — see the source link on each scoreFigures that model makers announced themselves, checked against the original post or system card and copied over. Conditions differ by maker, and each score links to its original source.
  • CyberGym leaderboard, UC Berkeley RDI (cybergym.io)From the leaderboard of UC Berkeley RDI, the group that made the test, only the rows its own team ran are included.