CritPt

Citation frequency 2/3
What does it measure?
Made by 50+ physicists led by Argonne National Laboratory and the University of Illinois, it tests solving physics problems at the level of early research projects rather than textbook exercises.
Who evaluates it, and how?
71 unpublished problems split into about 190 checkpoint steps. Answers are numerical arrays, formulas or Python functions, checked by an automatic grading program.
Reading the score
Scores run from 0 to 100%. Higher is better. Scores are very low across the board.
Top score36.0%
Models tested87
Scoring basisHigher is better
Model release date

Can it solve research-level physics problems?

CritPt

  1. 🥇GoogleGemini 4 CarbonReasoning effort: High—36.0%—
  2. 🥈AnthropicClaude Fable 5.2Reasoning effort: Extra High—35.7%1507
  3. 🥉OpenAIGPT-5.6 SolReasoning effort: Max49.5%32.3%1494
  4. #4AnthropicClaude Opus 5.5Reasoning effort: Max61.4%31.7%1517
  5. #4OpenAIGPT-6 AstraReasoning effort: Max54.7%31.7%1477
  6. #4OpenAIGPT-6.1 SolReasoning effort: Max52.9%31.7%1499
  7. #7AnthropicClaude Sonnet 5.5Reasoning effort: Max55.0%31.4%1497
  8. #8AnthropicClaude Fable 5.1Reasoning effort: Extra High59.1%31.1%1530
  9. #9OpenAIGPT-6 SolReasoning effort: Max47.9%30.9%1444
  10. #10OpenAIGPT-5.6 TerraReasoning effort: Max42.9%30.0%1478
  11. #10OpenAIGPT-5.4 ProReasoning effort: Extra High—30.0%—
  12. #12AnthropicClaude Opus 5Reasoning effort: Max54.9%29.1%1516
  13. #13AnthropicClaude Fable 5Reasoning effort: Max55.5%28.6%1521
  14. #14OpenAIGPT-6.1 TerraReasoning effort: Max46.2%28.0%1479
  15. #15GoogleGemini 4 ArgonReasoning effort: High57.1%27.1%1535
  16. #15OpenAIGPT-5.5Reasoning effort: Extra High45.8%27.1%1500
  17. #17XiaomiMiMo-V2.6-Pro49.4%26.6%1482
  18. #18MetaMuse Spark 1.3Reasoning effort: Extra High48.7%26.0%1502
  19. #19Moonshot AIKimi K3.1 Pro52.5%24.6%1477
  20. #20DeepSeekDeepSeek V4.1 Flash 1017Reasoning effort: Max43.3%24.0%—
  21. #21OpenAIGPT-5.4Reasoning effort: Extra High43.7%23.4%1491
  22. #22Moonshot AIKimi K3Reasoning effort: Max46.9%23.4%1502
  23. #23AnthropicClaude Haiku 5.5Reasoning effort: Extra High44.4%22.6%1447
  24. #24AnthropicClaude Opus 4.8Reasoning effort: Max48.7%20.9%1475
  25. #24StepfunStep-546.5%20.9%1474
  26. #24Z.aiGLM 5.2Reasoning effort: Max41.1%20.9%1488
  27. #27OpenAIGPT-5.6 LunaReasoning effort: Max39.5%20.6%1476
  28. #28SpaceXSIGrok 4.6Reasoning effort: Extra High44.1%19.7%1445
  29. #29OpenAIGPT-6 LunaReasoning effort: Max38.5%19.4%1449
  30. #30Z.aiGLM 5.3Reasoning effort: Max42.3%19.1%1504
  31. #31DeepSeekDeepSeek V4 ProReasoning effort: High41.0%18.6%1443
  32. #32GoogleGemini 3.8 FlashReasoning effort: High47.8%18.3%1519
  33. #33SpaceXSIGrok 4.7Reasoning effort: High43.1%18.0%1443
  34. #34GoogleGemini 3.1 Pro47.0%17.7%1488
  35. #34MetaMuse Spark 1.2Reasoning effort: Extra High45.5%17.7%1483
  36. #36AnthropicClaude Sonnet 5Reasoning effort: Max41.3%16.9%1472
  37. #37DeepSeekDeepSeek V4 FlashReasoning effort: Max38.6%16.6%1424
  38. #38SpaceXSIGrok 4.5Reasoning effort: High42.7%15.4%1471
  39. #38Z.aiGLM 5.3 Flash39.9%15.4%1509
  40. #40MetaMuse Spark 1.1Reasoning effort: Extra High46.2%15.1%1494
  41. #41GoogleGemini 3.7 FlashReasoning effort: High47.9%14.3%1501
  42. #41DeepSeekDeepSeek V4.1 FlashReasoning effort: Max39.2%14.3%1496
  43. #43AlibabaQwen3.7 Max40.5%13.4%1481
  44. #44GoogleGemini 3.5 FlashReasoning effort: High42.7%13.1%1482
  45. #45OpenAIGPT-5Reasoning effort: High28.5%12.6%1435
  46. #46AnthropicClaude Opus 4.7Reasoning effort: Max42.3%12.0%1490
  47. #46XiaomiMiMo-V2.6-Flash35.1%12.0%1463
  48. #48MetaMuse Spark40.7%11.3%1468
  49. #49Mistral AIMistral Large 4Reasoning effort: Medium35.0%11.1%1448
  50. #50GoogleGemini 3.6 FlashReasoning effort: High40.8%10.6%1501
  51. #51Moonshot AIKimi K2.7 Code35.0%10.0%—
  52. #51OpenAIGPT-5.4 MiniReasoning effort: Extra High28.1%10.0%1439
  53. #53OpenAIGPT-5.4 NanoReasoning effort: Extra High28.3%9.3%1424
  54. #54AlibabaQwen3.7 Plus35.6%9.1%1465
  55. #54SpaceXSIGrok Build 0.1—9.1%—
  56. #56Moonshot AIKimi K2.637.5%8.0%1479
  57. #56SpaceXSIGrok 4.3Reasoning effort: High37.2%8.0%1419
  58. #58Z.aiGLM-5.130.1%4.6%1474
  59. #59XiaomiMiMo V2.5 Pro35.7%4.0%1479
  60. #60MiniMaxMiniMax M339.0%3.7%1423
  61. #60XiaomiMiMo V2.527.2%3.7%1437
  62. #60InclusionAIRing-2.6-1T21.6%3.7%—
  63. #63AnthropicClaude Sonnet 4.6Reasoning effort: Max33.6%3.1%1461
  64. #63Moonshot AIKimi K2.530.7%3.1%1470
  65. #63NVIDIANemotron 3 Ultra28.4%3.1%1442
  66. #63NVIDIANemotron 3 Super20.8%3.1%1370
  67. #67AlibabaQwen3.6 Plus27.8%2.9%1451
  68. #68StepfunStep 3.7 Flash21.4%2.3%—
  69. #69GoogleGemini 2.5 Pro22.5%2.0%1439
  70. #70GoogleGemma 4 31B23.6%1.4%1466
  71. #71OpenAIGPT OSS 120BReasoning effort: High19.6%1.1%1381
  72. #71AnthropicClaude Sonnet 4.517.8%1.1%1426
  73. #71GoogleGemini 3.1 Flash Lite17.2%1.1%1436
  74. #74GoogleGemini 2.5 Flash12.1%1.1%1405
  75. #75AlibabaQwen3.5 397B A17BReasoning effort: None29.0%0.9%1451
  76. #75Arcee AITrinity Large Thinking15.8%0.9%1383
  77. #77MiniMaxMiniMax M2.729.6%0.6%1423
  78. #78AnthropicClaude Opus 412.3%0.3%1421
  79. #79AlibabaQwen3.6 35B A3B22.2%0.3%—
  80. #79AnthropicClaude Sonnet 410.7%0.3%1404
  81. #81OpenAIGPT-5 MiniReasoning effort: High21.5%0.0%1407
  82. #81GoogleGemini 3.5 Flash-Lite18.8%0.0%1442
  83. #81Mistral AIMistral Medium 3.513.8%0.0%1427
  84. #81AnthropicClaude Haiku 4.510.4%0.0%1402
  85. #81UpstageSolar Pro 310.3%0.0%—
  86. #81MetaLlama 4 Maverick4.9%0.0%1317
  87. #81MetaLlama 4 Scout3.8%0.0%1310
Hard test · top score 36.0% · Half of models ≤ 13.1%

The "harness" is the agent program the AI used to carry out the task. The same model can score very differently depending on its harness and reasoning effort.