Arena Japanese

In plain words · Which AI people preferred when asked in Japanese

What does it measure?

The ranking computed only on prompts written in Japanese. It can expose AIs that score well in English but fall short in Japanese expression and context.

Who evaluates it, and how?

On Arena Intelligence, a user enters a question, two anonymous AIs answer, and the user votes for the better one. Over 82 million votes are aggregated with a statistical model (Bradley-Terry) into an Elo-style score, and this table takes the 'style-controlled' version from the official dataset, which statistically removes the bias toward long, nicely formatted answers. The prompt language is detected automatically and only Japanese prompts are counted.

Reading the score

This is a head-to-head record, not an exam score (accuracy %). It is computed from the probability of being preferred over other AIs, so there is no maximum, and when a new AI enters, existing scores shift slightly. Typical values are 1,000–1,500; a 100-point gap means the leader is preferred about 64% of the time, a 200-point gap about 76%. Correctness is not counted — a wrong but friendly answer can win — so if accuracy matters, also look at accuracy tests (GPQA, AIME, etc.). Gaps of a few dozen points are effectively the same tier even if the rank differs. The sample is smaller than the English overall ranking, so ranks move more, and Japanese rankings correlate weakly with other languages — treat them separately.

Last updated: 2026-10-02