Arena Multi-turn

In plain words · Which AI people preferred in longer back-and-forth conversations

What does it measure?

Preference when a conversation continues over several turns rather than a single exchange — whether the AI remembers earlier context and stays consistent. Close to the real chat experience.

Who evaluates it, and how?

On Arena Intelligence, a user enters a question, two anonymous AIs answer, and the user votes for the better one. Over 82 million votes are aggregated with a statistical model (Bradley-Terry) into an Elo-style score, and this table takes the 'style-controlled' version from the official dataset, which statistically removes the bias toward long, nicely formatted answers. Only conversations with two or more exchanges are counted.

Reading the score

This is a head-to-head record, not an exam score (accuracy %). It is computed from the probability of being preferred over other AIs, so there is no maximum, and when a new AI enters, existing scores shift slightly. Typical values are 1,000–1,500; a 100-point gap means the leader is preferred about 64% of the time, a 200-point gap about 76%. Correctness is not counted — a wrong but friendly answer can win — so if accuracy matters, also look at accuracy tests (GPQA, AIME, etc.). Gaps of a few dozen points are effectively the same tier even if the rank differs. Useful for separating AIs that only give a good first answer from those that stay coherent as the conversation grows.

Last updated: 2026-10-02