NVIDIA Unveils 4GB Agent Model

NVIDIA announced on the 23rd local time the open-source lightweight language model Nemotron 3.5 Bolt E4B 4GB, designed to run with about 4GB of memory. It reduces memory use so it can run alongside speech and vision models on Windows PCs, in edge computing environments, and in robotics development. The model is designed to balance performance and memory use for game and robotics developers. Based on Google Gemma 4 E4B, it uses NVIDIA post-training techniques and runs on GeForce RTX, Jetson, RTX Spark, and DGX Spark.

On major conversational-agent benchmarks, it scored about 40% higher on average than the previous Nemotron 3 Nano 4B and about 10% higher on average than comparison models. In comparisons with reasoning enabled, it reduced output tokens by up to 80%. NVIDIA says generating fewer text tokens can reduce computing demand and cost while improving response speed. The model did not lead in every evaluation: it scored higher than competing models on 15 of 21 benchmarks, while Qwen3.5-4B led on some long-context and tool-calling evaluations. On Tau2-Bench, Bolt scored 55.1 and Qwen3.5-4B scored 75.1. The article says performance needs to be checked in real-world deployment settings.

Benchmark results and time to first token (TTFT) were measured as averages across six major agent evaluations on an RTX 5090. Local models used Q4_K_M quantization, and their performance and speed were compared with cloud models. The model can call APIs during conversations to perform tasks and sustain multiple exchanges; NVIDIA also provides a guide to asynchronous tool calls. It includes a short-answer mode for voice interfaces and reasoning that can be switched on or off according to request complexity. Reasoning time and output length can be limited to meet latency requirements.

The context window supports up to 64,000 tokens and can process conversation history, system instructions, tool definitions, search results, and application state together. The model supports 35 languages, while its base model was pretrained on more than 140 languages. Proposed uses include voice assistants on PCs, mobile devices, smart homes, and in vehicles; game NPCs and virtual companions that combine conversation with actions; and embedded or industrial agents for controlling robots and heavy equipment. NVIDIA says local execution can reduce reliance on external servers and ease constraints involving memory, latency, compatibility, operating costs, and privacy.

Post-training used reinforcement learning from verifiable rewards (RLVR) and reinforcement learning from human feedback (RLHF), with the NeMo-RL and NeMo-Gym frameworks. Training covered tool calling, instruction following, structured output, and multi-turn conversations, with reasoning and non-reasoning data in a 2-to-1 ratio. NVIDIA selected a BF16 checkpoint after training, then quantized it to Q4_K_M; importance-matrix calibration used about 1 million tokens of reinforcement-learning data. The company also released a DFlash draft generator for parallel inference. It creates multiple candidate tokens in one forward pass for verification by the target model in parallel. NVIDIA says this can increase throughput by up to 21% compared with Gemma 4 E4B's existing draft generator. The model is available through Hugging Face, Ollama, LM Studio, Unsloth, and Jetson AI Lab, and can be applied to game characters with an in-game inference SDK.

NVIDIA announced on the 23rd local time the open-source lightweight language model Nemotron 3.5 Bolt E4B 4GB, designed to run with about 4GB of memory. It reduces memory use so it can run alongside speech and vision models on Windows PCs, in edge computing environments, and in robotics development. The model is designed to balance performance and memory use for game and robotics developers. Based on Google Gemma 4 E4B, it uses NVIDIA post-training techniques and runs on GeForce RTX, Jetson, RTX Spark, and DGX Spark.

On major conversational-agent benchmarks, it scored about 40% higher on average than the previous Nemotron 3 Nano 4B and about 10% higher on average than comparison models. In comparisons with reasoning enabled, it reduced output tokens by up to 80%. NVIDIA says generating fewer text tokens can reduce computing demand and cost while improving response speed. The model did not lead in every evaluation: it scored higher than competing models on 15 of 21 benchmarks, while Qwen3.5-4B led on some long-context and tool-calling evaluations. On Tau2-Bench, Bolt scored 55.1 and Qwen3.5-4B scored 75.1. The article says performance needs to be checked in real-world deployment settings.

Benchmark results and time to first token (TTFT) were measured as averages across six major agent evaluations on an RTX 5090. Local models used Q4_K_M quantization, and their performance and speed were compared with cloud models. The model can call APIs during conversations to perform tasks and sustain multiple exchanges; NVIDIA also provides a guide to asynchronous tool calls. It includes a short-answer mode for voice interfaces and reasoning that can be switched on or off according to request complexity. Reasoning time and output length can be limited to meet latency requirements.

The context window supports up to 64,000 tokens and can process conversation history, system instructions, tool definitions, search results, and application state together. The model supports 35 languages, while its base model was pretrained on more than 140 languages. Proposed uses include voice assistants on PCs, mobile devices, smart homes, and in vehicles; game NPCs and virtual companions that combine conversation with actions; and embedded or industrial agents for controlling robots and heavy equipment. NVIDIA says local execution can reduce reliance on external servers and ease constraints involving memory, latency, compatibility, operating costs, and privacy.

Post-training used reinforcement learning from verifiable rewards (RLVR) and reinforcement learning from human feedback (RLHF), with the NeMo-RL and NeMo-Gym frameworks. Training covered tool calling, instruction following, structured output, and multi-turn conversations, with reasoning and non-reasoning data in a 2-to-1 ratio. NVIDIA selected a BF16 checkpoint after training, then quantized it to Q4_K_M; importance-matrix calibration used about 1 million tokens of reinforcement-learning data. The company also released a DFlash draft generator for parallel inference. It creates multiple candidate tokens in one forward pass for verification by the target model in parallel. NVIDIA says this can increase throughput by up to 21% compared with Gemma 4 E4B's existing draft generator. The model is available through Hugging Face, Ollama, LM Studio, Unsloth, and Jetson AI Lab, and can be applied to game characters with an in-game inference SDK.