SolarRenderer Renders Structured Video in Four Steps

SolarRenderer adapts a frozen MiniMax-H3 33B video backbone with two lightweight LoRAs to turn structural inputs into realistic video, according to an arXiv paper published on Oct 20. It can render in four steps, compared with 50-step sampling, and the paper reports that this is 12.3 times faster. The system uses frame-aligned training data so its output follows the structure while filling in appearance details.

The model sends structural video through the backbone’s reference pathway at the positions of the frames it describes. The authors say a single control LoRA can follow arbitrary structural representations without a type identifier; the inputs are described only in plain text. A block-causal variant supports streaming and renders 768p video in real time. The training corpus contains 136K clips from game engines, simulators and YouTube videos.

The authors report state-of-the-art results on both an in-distribution test set and a generalization benchmark covering 16 structural representations. SolarRenderer also generalizes to 11 representations absent from training, including pose skeletons, drawings and point clouds. A project-page description says the streaming version produces a 17-frame chunk every 0.70 seconds at 24 fps, about 1.1 seconds behind its input; with a KV cache, it can continue as long as the input does. On the held-out benchmark, the project page reports an Elo score of 1313 for structure, ranking first against nine other methods.

SolarRenderer adapts a frozen MiniMax-H3 33B video backbone with two lightweight LoRAs to turn structural inputs into realistic video, according to an arXiv paper published on Oct 20. It can render in four steps, compared with 50-step sampling, and the paper reports that this is 12.3 times faster. The system uses frame-aligned training data so its output follows the structure while filling in appearance details.

The model sends structural video through the backbone’s reference pathway at the positions of the frames it describes. The authors say a single control LoRA can follow arbitrary structural representations without a type identifier; the inputs are described only in plain text. A block-causal variant supports streaming and renders 768p video in real time. The training corpus contains 136K clips from game engines, simulators and YouTube videos.

The authors report state-of-the-art results on both an in-distribution test set and a generalization benchmark covering 16 structural representations. SolarRenderer also generalizes to 11 representations absent from training, including pose skeletons, drawings and point clouds. A project-page description says the streaming version produces a 17-frame chunk every 0.70 seconds at 24 fps, about 1.1 seconds behind its input; with a KV cache, it can continue as long as the input does. On the held-out benchmark, the project page reports an Elo score of 1313 for structure, ranking first against nine other methods.