All Benchmarks/Knowledge

JamC-QA

JapaneseHigher is better

A Japanese question-answering test on knowledge specific to Japan. It shows how much a model knows about Japan, which translated English tests do not reveal. Scores run from 0 to 100%. Higher is better.

Other versions:Japanese MT-Bench (Swallow)M-IFEval-JaPolyMath (Japanese, HT)Swallow Leaderboard — English (avg)Swallow Leaderboard — Japanese (avg)
Top score89.1%GPT-5.4
Models tested8
Model release date (newest on the right)
Best score so farHigher on the chart is better
  1. 1
    OpenAI
    GPT-5.4
    2026.03.05 · medium
    89.1%
  2. 2
    OpenAI
    GPT-5
    2025.08.07 · medium
    85.8%
  3. 3
    OpenAI
    GPT-4.1
    2025.04.14
    79.0%
  4. 4
    OpenAI
    GPT-5 Mini
    2025.08.07 · medium
    70.1%
  5. 5
    Google
    Gemma 4 31B
    2026.04.02
    68.6%
  6. 6
    NVIDIA
    Nemotron 3 Super
    2026.03.11
    59.3%
  7. 7
    Meta
    Llama 4 Scout
    2025.04.05
    57.9%
  8. 8
    OpenAI
    GPT OSS 120B
    2025.08.05 · medium
    51.8%

The "harness" is the agent program the AI used to carry out the task. The same model can score very differently depending on its harness and reasoning effort.