All Benchmarks/Knowledge

KMMLU-Pro

Often citedKoreanHigher is better

Multiple-choice questions from 14 Korean national professional licensing exams, such as medicine, law, tax, and accounting. The paper also reports pass/fail per license, but the value here is plain accuracy. Horangi runs a fixed 100-question subset rather than the full dataset, so its numbers are not directly comparable with full-dataset scores. Scores run from 0 to 100%. Higher is better.

Other versions:HAE-RAE Bench v1 (Reading)Horangi 4 — ALT (alignment)Horangi 4 — GLP (general language)Horangi 4 (Korean LLM Leaderboard)HRM8KIFEval-KoKMMLUKo-AIME 2025Ko-ARC-AGIKo-HLEKo-MoralKo-MTBenchKo-TruthfulQAKoBALT-700 (semantics)KoBALT-700 (syntax)KoBBQ (bias)Korean Hate Speech
Top score95.0%Claude Opus 4.8
Models tested48
Last updated2026.08.03
Model release date (newest on the right)
Best score so farHigher on the chart is better
  1. 1
    Anthropic
    Claude Opus 4.8
    2026.05.27 · high-effort
    95.0%
  2. 1
    OpenAI
    GPT-5.6 Sol
    2026.07.09 · max-effort
    95.0%
  3. 3
    OpenAI
    GPT-5.5
    2026.04.23 · xhigh-effort
    94.0%
  4. 4
    Google
    Gemini 3 Flash
    2025.12.17 · high-effort
    93.0%
  5. 5
    OpenAI
    GPT-5.6 Terra
    2026.07.09 · max-effort
    91.0%
  6. 5
    OpenAI
    GPT-5.4
    2026.03.05 · xhigh-effort
    91.0%
  7. 7
    Anthropic
    Claude Sonnet 5
    2026.06.30 · high-effort
    90.0%
  8. 7
    OpenAI
    GPT-5.6 Luna
    2026.07.09 · max-effort
    90.0%
  9. 9
    Google
    Gemini 3.5 Flash
    2026.05.19
    89.9%
  10. 9
    Google
    Gemini 3.1 Pro
    2026.02.19 · high-effort
    89.9%

The "harness" is the agent program the AI used to carry out the task. The same model can score very differently depending on its harness and reasoning effort.