Nucleus sampling

What is Nucleus sampling?

A decoding method that restricts the model's choice of next tokens to the smallest possible set whose cumulative probability exceeds a predefined threshold (top-p).

In other languages

한국어뉴클리어스 샘플링

모델이 생성할 다음 단어의 확률 분포 중 누적 확률이 특정 임계값(Top-p)에 도달할 때까지만 후보를 선택하는 확률적 샘플링 방식

日本語Top-pサンプリング

累積確率が一定値（p値）に達するまでの単語を選択肢として抽出する生成時の確率制御手法

Related Terms

• Constrained DecodingA technique that restricts a language model to select only tokens conforming to specific grammar or format rules when predicting the next word.
• Speculative decodingA text generation technique where a small, fast model proposes candidate tokens first, and a larger model verifies them, improving generation speed.
• Probabilistic ModelA statistical framework that predicts the likelihood of an outcome or the next token in a sequence based on training data patterns
• Schema-constrained decodingGeneration method that restricts possible output tokens so responses follow a required structure
• Parallel DecodingA text generation method that produces multiple tokens in a single step, reducing latency.