EmoReward Calibrates Rewards for Video Generation

Tencent Hunyuan researchers published EmoReward on November 4, proposing a self-evolving training method to improve emotional expression in generated video. Their paper, submitted on November 5, addresses the difficulty of scoring emotional video quality, which depends on subjective aesthetics and lacks a uniquely correct target. Existing methods often use fixed rewards for visual quality, motion quality, and text alignment; the authors say these can miss emotional intent and invite reward hacking. They also describe reinforcement learning from human feedback (training guided by human preferences) as difficult to scale, with individual aesthetic differences complicating how facial expression, body motion, and color cues are judged across narrative contexts.

EmoReward uses reward traces collected during a training run to estimate which emotional evidence is reliable, then adjusts the weights assigned to multiple rewards in a later run. It first plans specific visual cues from an emotional prompt and uses different signal types to score the generated video. Reflective Reward Gating adjusts each signal’s contribution using instance-level gates for phase, face confidence, conflict, and subtlety, then renormalizes weights across the active signals to calculate a combined reward for each sample.

RewardMemory estimates each signal type’s reliability from rollout scores and outcomes within each emotional regime in a completed training run. It then sets bounded multipliers to calibrate reward weights in the next run, without updating reward models. Experiments with HunyuanVideo 1.5 showed improvements across multiple reinforcement learning algorithms; under SAGE-GRPO, the researchers compared EmoReward with other agentic reward methods. Ablation and transfer experiments using SD 3.5 and FLUX further supported adaptive reward calibration for visual generation, according to the paper.

Tencent Hunyuan researchers published EmoReward on November 4, proposing a self-evolving training method to improve emotional expression in generated video. Their paper, submitted on November 5, addresses the difficulty of scoring emotional video quality, which depends on subjective aesthetics and lacks a uniquely correct target. Existing methods often use fixed rewards for visual quality, motion quality, and text alignment; the authors say these can miss emotional intent and invite reward hacking. They also describe reinforcement learning from human feedback (training guided by human preferences) as difficult to scale, with individual aesthetic differences complicating how facial expression, body motion, and color cues are judged across narrative contexts.

EmoReward uses reward traces collected during a training run to estimate which emotional evidence is reliable, then adjusts the weights assigned to multiple rewards in a later run. It first plans specific visual cues from an emotional prompt and uses different signal types to score the generated video. Reflective Reward Gating adjusts each signal’s contribution using instance-level gates for phase, face confidence, conflict, and subtlety, then renormalizes weights across the active signals to calculate a combined reward for each sample.

RewardMemory estimates each signal type’s reliability from rollout scores and outcomes within each emotional regime in a completed training run. It then sets bounded multipliers to calibrate reward weights in the next run, without updating reward models. Experiments with HunyuanVideo 1.5 showed improvements across multiple reinforcement learning algorithms; under SAGE-GRPO, the researchers compared EmoReward with other agentic reward methods. Ablation and transfer experiments using SD 3.5 and FLUX further supported adaptive reward calibration for visual generation, according to the paper.