A reward system learns which emotional video cues to trust
EmoReward: Self-Evolving Reward Calibration for Emotional Video Generation RL · arXiv:2611.06518
In plain words
Emotional video generators need a way to judge whether a clip expresses a prompt well. EmoReward checks which clues are available in each video, then uses experience from one completed training run to adjust how much those clues count in the next run. It does not change the scoring models.
Across three video-training methods, guided versions scored higher than their matching baselines on the paper's video scorer. Human ratings also favored EmoReward on several measures, though the tests do not show that this advantage continues across repeated training runs.
The research
A video may look polished and follow its prompt without conveying the intended emotion, while the clues that signal emotion vary from scene to scene.
These results change the case for fixed reward mixtures: in the tested video settings, rewards that screen cues per video and adjust their weights from prior-run evidence scored better on VideoAlign.
EmoReward gates emotional evidence within each video and uses provider reliability from a completed run to adjust reward weights in the next video-training run.
In the main comparison, HunyuanVideo 1.5 was trained with three video reinforcement-learning algorithms, then guided configurations were compared with their corresponding optimizer baselines on 302 emotional prompts.
VideoAlign scores rose across all four of its measures in each comparison, supporting the use of context-sensitive reward guidance in these tested settings.
A separate seven-policy comparison favored the version with memory on two defined composite diagnostics. Human ratings also gave it larger Good shares than SAGE-GRPO, while image-transfer results varied by setting.
These different measures support adaptive reward allocation, but VideoAlign agreement is not independent evidence of emotional fidelity.
The calibration signal partly shares information with VideoAlign, and the reported memory experiments test one transition from a reference run to a later run. The results do not show whether repeated transitions keep improving emotional fidelity.
What the evidence shows
- Emotional cues vary by scene
EmoReward is a training mechanism that adjusts reward-provider weights for emotional video generation. The problem is that no single visual target defines a correct emotional video, and clues such as facial expression or motion may be missing or misleading in a given scene. Fixed reward mixtures can therefore overvalue a cue that does not fit the current video.
- Guided rewards score higher
The main comparison uses VideoAlign, which scores generated videos for overall result, visual quality, motion quality and text alignment. On 302 emotional prompts outside training, DanceGRPO with Reflective Reward Gating and FlowGRPO and SAGE-GRPO with EmoReward each scored higher than their matching optimizer baseline on all four measures. The result favors guided configurations on this evaluator across these three tested algorithms.
- Memory leads on diagnostics
In the seven-policy SAGE-GRPO comparison, adding RewardMemory gives EmoReward the strongest standing on the defined composite diagnostics. Because those scores are scaled within this comparison and include agreement with VideoAlign, they do not establish independent emotional fidelity.
- Independent checks are mixed
Human ratings provide a separate, descriptive view: EmoReward received larger Good shares than SAGE-GRPO across the three rated criteria.
Image-generation transfer was mixed across settings, so these results support adaptive reward allocation in tested settings, not uniform improvement across measures.
Deep
- Original paper authors
- Mingzhe Zheng, Yue Wu, Jing Tan, Chang Zou, Dengyang Jiang
- Submitted to arXiv
- License
- CC BY 4.0