Japanese MT-Bench (Swallow)
JapaneseHigher is better
Japanese MT-Bench as run by the Swallow leaderboard itself. Models answer questions over two turns across eight areas (writing, roleplay, reasoning, math, coding, extraction, STEM, humanities) and an AI judge grades them. The questions come from the Nejumi edition but grading is done separately, so values differ from Nejumi's Japanese MT-Bench score. Scores run from 0 to 100%. Higher is better.
Top score84.4%GPT-5.4
Models tested8
Model release date (newest on the right)
Best score so farHigher on the chart is better
RankModelSettingReleasedPrice/1MScore
- 1medium2026.03.05$1284.4%
- 2medium2025.08.07$7.8184.2%
- 3medium2025.08.07$1.5683.0%
- 42026.04.02$0.3481.5%
- 52025.04.14$6.5077.1%
- 6medium2025.08.05$0.4075.7%
- 72026.03.11$0.3473.6%
- 82025.04.05$0.3762.9%
The "harness" is the agent program the AI used to carry out the task. The same model can score very differently depending on its harness and reasoning effort.