Decision AI in Korean and Japanese: Near-Frontier Results at Lower Cost

In an AI answer-review test, Jev 1.13 finished just 2.0 percentage points behind GPT-6.1 Sol. Across the full evaluation, its estimated processing time was about nine times shorter and its recorded response cost about one thirty-eighth as much. For Korean and Japanese workflows, these results deserve a closer look.
AIB's DecisionBench-KJ compares decision-focused AI in both languages. It shows where specialist models approach leading general-purpose AI, and where differences in language, task, speed and cost could change your choice.
A benchmark for Korean and Japanese work
Decision AI returns a choice, probability or score for questions with defined answers: whether an inquiry concerns shipping, refunds or payment, for example, or whether a response agrees with its source.
The principles are covered in The AI Opposite of ChatGPT: Why Is the New Model “Jev” Getting So Much Attention?, and the business implications in Decision AI After Jev: What OpenAI and Laya Could Change for Businesses. Here are the measured results.
Version 0.2, as of October 4, 2026, compares seven decision models and two general-purpose baselines using the same 2,000 questions from eight public dataset families. The baselines are GPT-6.1 Sol and Claude Sonnet 5.5. Six decision models completed the evaluation; Mercury Decide remains incomplete and is excluded from the rankings.
Four tasks close to frontier performance
“Frontier models” are general-purpose AI models competing at the highest performance levels. In this evaluation, decision models matched or approached them on several tasks:
- Reviewing AI answers: On Korean and Japanese M-RewardBench questions, which ask models to choose the better of two answers, Jev 1.13 scored 86.4%, versus 88.4% for GPT-6.1 Sol
- Checking consistency with a source: Across Korean KLUE NLI and Japanese JGLUE JNLI, Perplexity Decider v1 27B scored 91.2%, matching Claude Sonnet 5.5
- Answering yes or no from a passage: On 100 Korean KoBEST BoolQ questions, Jev scored 100%, versus 99% for GPT-6.1 Sol
- Classifying positive and negative sentiment: On 60 Korean KoBEST SentiNeg questions, Perplexity Decider v1 27B, Solar Decide and D1 each scored 98.3%, matching GPT-6.1 Sol
These public-data results help identify tasks worth testing, such as reviewing answer candidates or classifying documents. Performance on your company's material still needs to be checked.
Japanese results change the ranking
The Japanese-language evaluation is particularly striking. Jev 1.13 scored 79.4, ahead of GPT-6.1 Sol at 78.6 and Claude Sonnet 5.5 at 73.9. This score equally weights accuracy across the five dataset families containing Japanese questions; it is not pooled question-level accuracy.
For a team handling Japanese customer inquiries, that makes Jev a candidate worth testing. It does not establish a lead on every Japanese task. The Korean results also vary by task, making the language and decisions your customers actually use a better starting point than a single overall ranking.
Jev: lower response time and cost
Across the full Korean-and-Japanese evaluation, Jev led the completed decision models with 78.6, compared with 83.9 for GPT-6.1 Sol: a gap of 5.3 points. This overall score is an equally weighted average of dataset accuracies.
Speed and cost add another dimension:
- Estimated time for 1,000 sequential requests: Jev took 3 minutes 45 seconds, versus 34 minutes 25 seconds for GPT-6.1 Sol, roughly a ninefold difference. These estimates multiply mean response time by 1,000
- Recorded response cost per 1,000 requests: Approximately US$0.022 for Jev and US$0.826 for GPT-6.1 Sol, putting Jev at about one thirty-eighth of the cost
Costs reflect stored responses; some failed requests before retries may be excluded. Operating budgets also need to include retries and human review.
A high-performance model could write the final customer reply while decision AI handles repeated classification and checks. Splitting the work where accuracy meets your requirements could reduce waiting time and cost.
Different models, different strengths
D1 had the lowest recorded cost among completed paid decision models: US$0.0075 per 1,000 requests. Temporarily free models are excluded from this comparison.
Perplexity Decider v1 27B had the best probability calibration among completed decision models. Its expected calibration error, measuring the gap between confidence and observed accuracy, was 2.3 percentage points. Calibration helps assess how much to trust a model's stated confidence when deciding which cases need human review.
For Korean hate-speech classification, Upstage's Solar Decide scored 83.6% on 250 K-MHaS questions, ahead of Jev's 75.6% and first among the six completed decision models. GPT-6.1 Sol scored 96.4% on that task, so a specialist's relative strength still needs to be weighed against the remaining accuracy gap.
Test the work your customers bring
Which recurring task needs just one choice before the next step can begin?
Use DecisionBench-KJ to shortlist candidates by use case. Then compare them on real examples with personal data minimized and answers agreed by your team. Keep Korean and Japanese results separate where both languages matter.
Alongside accuracy, record important cases missed, response time and staff correction time. Start where the model reduces review work. Keep human approval for consequential decisions such as refunds and contracts.
AIB DecisionBench-KJ: a Korean & Japanese benchmark for decision models | AIB
Put AI to work with AIB
AIB is a startup operating an independent AI comparison platform, without favoring a particular AI company. We aim to help businesses choose AI that fits their work and budget, so differences in information and cost do not become barriers to adoption.
If you are choosing AI for recurring work or looking to reduce costs and waiting time, tell us about your needs.
Contact AIB about AI adoption, costs and practical use
Sources and further reading
- Direct measurements and comparisons: AIB DecisionBench-KJ: a Korean & Japanese benchmark for decision models | AIB

Teach Kids About AI, Limit Their AI Use: 6 Family Rules for Safe AI
In our last post, we looked at "What Should Kids Learn When AI Does the Work?" This time, let's look at what families can actually do at home. O…

How to Adopt AI at Work: Define Requirements, Test and Monitor Results
A team agrees to automate customer support with AI. The meeting ends, but one question remains: what exactly is the AI supposed to do? Some peop…

Who Will Control AI Agents: The AI Builders or the Platforms Holding the Data and Tools?
When a shopper tells an AI, “Find the right one and buy it for me,” who controls the customer relationship? The AI company that understands the…

How Do You Build and Connect AI Agents? Notes from Google Agentic AI Night
How do you build AI agents and connect them so they can take on a task together? Start by defining each agent's role, the instructions it should…