NVIDIA Releases Compact Speech Restoration AI

NVIDIA released its general-purpose speech enhancement model, NemotronLabs RE-USE, on Hugging Face on October 30 local time. With 9.6 million parameters, it is designed to reduce noise, reverberation, clipped speech, bandwidth limits, codec damage, packet loss, and distortion from low-quality microphones while preserving speech clarity and speaker identity. The commercially usable open model supports real-time processing across multiple recording environments and languages.

RE-USE scored higher on benchmarks than Sidon, with 250 million parameters, and UniPASE, with 500 million. Its model weights occupy 38.4 MB in FP32. It accepts audio sampled from 8 kHz to 48 kHz and processes 8 kHz, 16 kHz, 22.05 kHz, 24 kHz, 32 kHz, 44.1 kHz, and 48 kHz audio without separate models. It was trained on English, German, Spanish, French, and Chinese speech and is intended to avoid dependence on a specific language.

When several people speak together, RE-USE is trained to preserve the main speaker’s voice and suppress other voices. It can make a particular speaker clearer in interviews or podcasts, but it cannot separate speakers individually or identify them, so a separate tool is needed. NVIDIA said RE-USE scored higher than Sidon and UniPASE on all four metrics in the URGENT 2025 non-blind test set: PESQ, STOI, speaker similarity, and speech recognition accuracy.

For 16 kHz speech data, word accuracy rose from 76.5% in noisy original audio to 87.1% after restoration with RE-USE. On 44.1 kHz data, speaker similarity was 0.90, compared with 0.57 for Resemble Enhance. The model combines bidirectional Mamba with a convolutional encoder and decoder; its central bidirectional Mamba core has 30 layers. Mamba is a state-space model sequence-processing architecture whose computation grows linearly with input length, while full self-attention computation grows quadratically with sequence length.

Training used about 2,500 hours of synthetic speech data and room and microphone impulse responses that reproduce acoustic properties of real spaces. Suggested uses include improving podcast and video audio, cleaning speech datasets, preprocessing for automatic speech recognition (ASR), and preparing training data for text-to-speech (TTS) and speech-based large language models. The official inference environment is based on Linux and NVIDIA Ampere GPUs, using Triton Inference Server acceleration on systems such as the A100. NVIDIA also provides a Hugging Face demo and a script that processes recordings longer than 30 seconds by splitting them into overlapping segments.

NVIDIA released its general-purpose speech enhancement model, NemotronLabs RE-USE, on Hugging Face on October 30 local time. With 9.6 million parameters, it is designed to reduce noise, reverberation, clipped speech, bandwidth limits, codec damage, packet loss, and distortion from low-quality microphones while preserving speech clarity and speaker identity. The commercially usable open model supports real-time processing across multiple recording environments and languages.

RE-USE scored higher on benchmarks than Sidon, with 250 million parameters, and UniPASE, with 500 million. Its model weights occupy 38.4 MB in FP32. It accepts audio sampled from 8 kHz to 48 kHz and processes 8 kHz, 16 kHz, 22.05 kHz, 24 kHz, 32 kHz, 44.1 kHz, and 48 kHz audio without separate models. It was trained on English, German, Spanish, French, and Chinese speech and is intended to avoid dependence on a specific language.

When several people speak together, RE-USE is trained to preserve the main speaker’s voice and suppress other voices. It can make a particular speaker clearer in interviews or podcasts, but it cannot separate speakers individually or identify them, so a separate tool is needed. NVIDIA said RE-USE scored higher than Sidon and UniPASE on all four metrics in the URGENT 2025 non-blind test set: PESQ, STOI, speaker similarity, and speech recognition accuracy.

For 16 kHz speech data, word accuracy rose from 76.5% in noisy original audio to 87.1% after restoration with RE-USE. On 44.1 kHz data, speaker similarity was 0.90, compared with 0.57 for Resemble Enhance. The model combines bidirectional Mamba with a convolutional encoder and decoder; its central bidirectional Mamba core has 30 layers. Mamba is a state-space model sequence-processing architecture whose computation grows linearly with input length, while full self-attention computation grows quadratically with sequence length.

Training used about 2,500 hours of synthetic speech data and room and microphone impulse responses that reproduce acoustic properties of real spaces. Suggested uses include improving podcast and video audio, cleaning speech datasets, preprocessing for automatic speech recognition (ASR), and preparing training data for text-to-speech (TTS) and speech-based large language models. The official inference environment is based on Linux and NVIDIA Ampere GPUs, using Triton Inference Server acceleration on systems such as the A100. NVIDIA also provides a Hugging Face demo and a script that processes recordings longer than 30 seconds by splitting them into overlapping segments.