PFN Releases PLaMo 3 Translate, a Translation-Specialized LLM

Preferred Networks (PFN) has released the weights of its translation-specialized LLM "PLaMo 3 Translate" on Hugging Face. It comes in four sizes: 0.8B, 2B, 8B, and 31B, and supports translation across 53 languages. According to the developer, even the smallest 0.8B model achieves performance in Japanese-English and multilingual translation that surpasses existing translation-specialized LLMs more than 30 times larger. It also supports surrounding context, translation glossaries, and specification of plain versus polite speech style.

For each size, PFN provides BF16 weights along with FP8, W4A16, W4AFP8, NVFP4, GGUF, and MLX formats. Quantization was performed using tools such as AutoRound, and the company states that benchmark accuracy loss for each quantized format was minimized relative to the BF16 version. Users can choose a distribution format according to their environment, such as supported GPUs, CPUs, Apple silicon, or inference engines. The license is the PLaMo Community License. Personal, learning, and hobby use is free, and non-commercial use requires no advance registration. If the user or an affiliated company has annual revenue of 1 billion yen or less in its most recent fiscal year, commercial use is also free after pre-registration through a dedicated form. Commercial use by entities exceeding 1 billion yen requires a separate contract.

PFN also demonstrated an internal demo running the 0.8B model on a smartphone. It performs fast, natural translation even without an internet connection, such as in airplane mode, though PFN does not currently offer this app itself.

Training consisted of two-stage supervised fine-tuning (learning from correct example outputs) applied to the pretrained base model of PLaMo's third generation, followed by direct preference optimization (a tuning method that makes preferred responses more likely to be selected). In the first stage, the model was adapted to translation using a large volume of source-translation sentence pairs. In the second stage, quality was improved using diverse source texts closer to real-world use and more refined synthetic translations. For models of 8B and below, knowledge distillation from the 31B model was performed between the second-stage supervised fine-tuning and preference optimization, transferring the capabilities of the larger model to the smaller ones.

For translation evaluation, PFN used its internal PFTB benchmark for Japanese-English and an English-Japanese SaaS benchmark that approximates real translation SaaS requests. Both measure string similarity against reference translations. Additionally, COMET-22 was used on the Japanese track of WMT24++, and chrF++ was used on FLORES+ for translation between Japanese, English, and multiple other languages. PFTB includes papers, product reviews, and poetry, while the SaaS benchmark requires translation of web pages while preserving HTML tags. The company notes that WMT24++ consists mainly of short sentences and has concerns about reference translation quality, so development primarily referred to PFTB and SaaS scores.

Most training data was created by collecting source texts from the web and similar sources, then generating translations with an LLM. During source collection, PFN avoided spam and sought diversity in writing style, host, and domain, prioritizing text written natively in each language. Text with language mixing or machine-translation artifacts was excluded using language-specific rules and filters. Gemma 4 31B IT was used for translation synthesis. A tournament-style data synthesis process compared and fused multiple candidates, incorporating the strengths of each candidate to improve quality. Back-translation pairs were also created, converting synthetic translations back into source text; to preserve information correspondence, Gemma 4 was used to adjust the source text while keeping natural expression as much as possible.

In iterative self-distillation, the current PLaMo translation model generates multiple candidates, and the translations improved through tournament-style synthesis are compared against the existing translations in the dataset. When the new translation is judged better, it is adopted into the data, and the model is retrained on the updated data, repeating this process. Due to computational cost, this method was applied only to second-stage data. The final dataset totals approximately 37.3B tokens: about 33.9B for the first stage and about 3.4B for the second stage, which the company describes as equivalent to several hundred thousand paperback books. The second stage includes academic papers, literature, technical documents, finance, patents, medicine, news, encyclopedias, web text, colloquial language, transcriptions, browser translation, and file translation.

According to the company's evaluation, Japanese-English performance improved substantially from the base model through first-stage supervised fine-tuning and further improved in the secon

Preferred Networks (PFN) has released the weights of its translation-specialized LLM "PLaMo 3 Translate" on Hugging Face. It comes in four sizes: 0.8B, 2B, 8B, and 31B, and supports translation across 53 languages. According to the developer, even the smallest 0.8B model achieves performance in Japanese-English and multilingual translation that surpasses existing translation-specialized LLMs more than 30 times larger. It also supports surrounding context, translation glossaries, and specification of plain versus polite speech style.

For each size, PFN provides BF16 weights along with FP8, W4A16, W4AFP8, NVFP4, GGUF, and MLX formats. Quantization was performed using tools such as AutoRound, and the company states that benchmark accuracy loss for each quantized format was minimized relative to the BF16 version. Users can choose a distribution format according to their environment, such as supported GPUs, CPUs, Apple silicon, or inference engines. The license is the PLaMo Community License. Personal, learning, and hobby use is free, and non-commercial use requires no advance registration. If the user or an affiliated company has annual revenue of 1 billion yen or less in its most recent fiscal year, commercial use is also free after pre-registration through a dedicated form. Commercial use by entities exceeding 1 billion yen requires a separate contract.

PFN also demonstrated an internal demo running the 0.8B model on a smartphone. It performs fast, natural translation even without an internet connection, such as in airplane mode, though PFN does not currently offer this app itself.

Training consisted of two-stage supervised fine-tuning (learning from correct example outputs) applied to the pretrained base model of PLaMo's third generation, followed by direct preference optimization (a tuning method that makes preferred responses more likely to be selected). In the first stage, the model was adapted to translation using a large volume of source-translation sentence pairs. In the second stage, quality was improved using diverse source texts closer to real-world use and more refined synthetic translations. For models of 8B and below, knowledge distillation from the 31B model was performed between the second-stage supervised fine-tuning and preference optimization, transferring the capabilities of the larger model to the smaller ones.

For translation evaluation, PFN used its internal PFTB benchmark for Japanese-English and an English-Japanese SaaS benchmark that approximates real translation SaaS requests. Both measure string similarity against reference translations. Additionally, COMET-22 was used on the Japanese track of WMT24++, and chrF++ was used on FLORES+ for translation between Japanese, English, and multiple other languages. PFTB includes papers, product reviews, and poetry, while the SaaS benchmark requires translation of web pages while preserving HTML tags. The company notes that WMT24++ consists mainly of short sentences and has concerns about reference translation quality, so development primarily referred to PFTB and SaaS scores.

Most training data was created by collecting source texts from the web and similar sources, then generating translations with an LLM. During source collection, PFN avoided spam and sought diversity in writing style, host, and domain, prioritizing text written natively in each language. Text with language mixing or machine-translation artifacts was excluded using language-specific rules and filters. Gemma 4 31B IT was used for translation synthesis. A tournament-style data synthesis process compared and fused multiple candidates, incorporating the strengths of each candidate to improve quality. Back-translation pairs were also created, converting synthetic translations back into source text; to preserve information correspondence, Gemma 4 was used to adjust the source text while keeping natural expression as much as possible.

In iterative self-distillation, the current PLaMo translation model generates multiple candidates, and the translations improved through tournament-style synthesis are compared against the existing translations in the dataset. When the new translation is judged better, it is adopted into the data, and the model is retrained on the updated data, repeating this process. Due to computational cost, this method was applied only to second-stage data. The final dataset totals approximately 37.3B tokens: about 33.9B for the first stage and about 3.4B for the second stage, which the company describes as equivalent to several hundred thousand paperback books. The second stage includes academic papers, literature, technical documents, finance, patents, medicine, news, encyclopedias, web text, colloquial language, transcriptions, browser translation, and file translation.

According to the company's evaluation, Japanese-English performance improved substantially from the base model through first-stage supervised fine-tuning and further improved in the secon