Arena Hard Prompts

In plain words · Which AI people preferred on the hardest questions

What does it measure?

The ranking recomputed on hard questions only, with easy ones removed. It shows which AI was preferred on questions needing expertise, complex reasoning and accuracy.

Who evaluates it, and how?

On Arena Intelligence, a user enters a question, two anonymous AIs answer, and the user votes for the better one. Over 82 million votes are aggregated with a statistical model (Bradley-Terry) into an Elo-style score, and this table takes the 'style-controlled' version from the official dataset, which statistically removes the bias toward long, nicely formatted answers. Prompts are rated on seven criteria — specificity, domain knowledge, reasoning steps, technical accuracy and more — and only the top third or so are counted.

Reading the score

This is a head-to-head record, not an exam score (accuracy %). It is computed from the probability of being preferred over other AIs, so there is no maximum, and when a new AI enters, existing scores shift slightly. Typical values are 1,000–1,500; a 100-point gap means the leader is preferred about 64% of the time, a 200-point gap about 76%. Correctness is not counted — a wrong but friendly answer can win — so if accuracy matters, also look at accuracy tests (GPQA, AIME, etc.). Gaps of a few dozen points are effectively the same tier even if the rank differs. An AI whose score differs a lot from the overall ranking performs differently on easy versus hard questions.

Last updated: 2026-10-02