AI rankings for coding

WeirdML

Higher is better

Unusual machine-learning tasks, such as shape recognition, odd digit classification, and predicting chess results: the AI writes PyTorch code, sees the results, and revises up to five times. Values here are from version 2 (19 tasks) and are not comparable with the current version 3 (11 tasks). Scores run from 0 to 100%. Higher is better.

Top score93.3%GPT-6 Astra
Models tested57
Model release date (newest on the right)
Best score so farHigher on the chart is better
  1. 🥇OpenAIGPT-6 AstraReasoning effort: Max93.3%
  2. 🥈AnthropicClaude Fable 5.1Reasoning effort: Max92.9%
  3. 🥉AnthropicClaude Fable 5Reasoning effort: Max91.9%
  4. #4AnthropicClaude Opus 5Reasoning effort: Max91.8%
  5. #5OpenAIGPT-5.6 SolReasoning effort: Max89.4%
  6. #6OpenAIGPT-5.5Reasoning effort: Extra High84.9%
  7. #7GoogleGemini 3.8 FlashReasoning effort: High84.8%
  8. #8AnthropicClaude Opus 4.8Reasoning effort: Extra High82.9%
  9. #9Moonshot AIKimi K3Reasoning effort: Max82.6%
  10. #10OpenAIGPT-5.6 TerraReasoning effort: High78.3%
  11. #11AnthropicClaude Opus 4.6Reasoning effort: High78.0%
  12. #12OpenAIGPT-5.4Reasoning effort: Extra High77.7%
  13. #13AnthropicClaude Opus 4.7Reasoning effort: High76.4%
  14. #14Z.aiGLM 5.3Reasoning effort: Max75.4%
  15. #15GoogleGemini 3.1 Pro72.1%
  16. #16Z.aiGLM 5.2Reasoning effort: Max70.1%
  17. #17AnthropicClaude Sonnet 5Reasoning effort: High68.8%
  18. #18xAIGrok 4.6Reasoning effort: High67.3%
  19. #19DeepSeekDeepSeek V4 ProReasoning effort: Max66.2%
  20. #20AnthropicClaude Sonnet 4.6Reasoning effort: Medium66.1%
  21. #21AnthropicClaude Opus 4.5Reasoning effort: Budget 16K63.7%
  22. #22DeepSeekDeepSeek V4 FlashReasoning effort: Max63.0%
  23. #23GoogleGemini 3.5 FlashReasoning effort: High62.6%
  24. #24GoogleGemini 3 Flash61.6%
  25. #25OpenAIGPT-5.6 LunaReasoning effort: High60.9%
  26. #26OpenAIGPT-5Reasoning effort: High60.7%
  27. #27MetaMuse Spark 1.2Reasoning effort: Extra High60.3%
  28. #27OpenAIGPT-5.4 MiniReasoning effort: High60.3%
  29. #29OpenAIGPT-5.4 ProReasoning effort: None57.4%
  30. #30Z.aiGLM-5.157.1%
  31. #31GoogleGemini 3.6 FlashReasoning effort: High56.1%
  32. #32Moonshot AIKimi K2.655.9%
  33. #33Moonshot AIKimi K2.7 Code54.1%
  34. #34GoogleGemini 2.5 ProReasoning effort: Budget 16K54.0%
  35. #35OpenAIGPT-5 MiniReasoning effort: High52.7%
  36. #36GoogleGemma 4 31B52.3%
  37. #37GoogleGemini 3.1 Flash Lite52.2%
  38. #38xAIGrok 4.349.9%
  39. #39OpenAIGPT-5.4 NanoReasoning effort: High49.2%
  40. #40OpenAIGPT OSS 120BReasoning effort: High48.2%
  41. #40Z.aiGLM-548.2%
  42. #42AnthropicClaude Sonnet 4.5Reasoning effort: Budget 16K47.7%
  43. #43xAIGrok 4.546.4%
  44. #44AnthropicClaude Sonnet 4Reasoning effort: Budget 16K46.1%
  45. #45AnthropicClaude Opus 4.1Reasoning effort: Budget 16K45.9%
  46. #46Moonshot AIKimi K2.545.6%
  47. #47AnthropicClaude Haiku 4.545.4%
  48. #48AnthropicClaude Opus 4Reasoning effort: Budget 16K43.7%
  49. #49Mistral AIMistral Medium 3.543.7%
  50. #50NVIDIANemotron 3 Ultra43.5%
  51. #51OpenAIGPT-4.139.0%
  52. #52GoogleGemini 3.5 Flash-LiteReasoning effort: High39.0%
  53. #53OpenAIGPT-5 NanoReasoning effort: High38.1%
  54. #54NVIDIANemotron 3 Super38.0%
  55. #55MiniMaxMiniMax M2.737.0%
  56. #56AlibabaQwen3.6 35B A3B34.5%
  57. #57MetaLlama 4 Maverick24.5%
Half of models ≤ 57.4%Last updated 2026-10-07

The "harness" is the agent program the AI used to carry out the task. The same model can score very differently depending on its harness and reasoning effort.