Reviso Prioritizes Evidence in Video Captions

Researchers proposed Reviso, an agentic video-captioning framework, in a paper published on November 3. The system aims to produce captions that emphasize key events and context instead of giving minor visual details equal weight. Experiments across four video-captioning benchmarks found that Reviso outperformed the evaluated open-source baselines, and its captions better supported downstream video question answering.

Reviso begins with a low-frame-rate preview of a video, then builds a hierarchical Context–Focus–Points (CFP) plan. The plan identifies the overall context, prioritizes important aspects, and sets specific targets and tools for closer inspection. Reviso selectively revisits video segments, combines tools to examine them in detail, and updates its plan as it interprets the evidence. It then combines information from the full video and selected segments to produce a caption.

To teach this observation strategy, the researchers created a training pipeline that checks factual accuracy and refines examples with guidance from a teacher model. They trained Reviso with Group Relative Policy Optimization (GRPO), using a key-fact reward for caption quality, a Plan Alignment Reward (PAR) for selecting important observation targets, and an Unproductive Tool-Call Penalty (UTCP). UTCP adjusts penalties for observation costs based on caption quality. The paper reports results against evaluated open-source systems and says Reviso’s captions also improved support for video question answering; it gives no benchmark scores in the provided abstract.

Researchers proposed Reviso, an agentic video-captioning framework, in a paper published on November 3. The system aims to produce captions that emphasize key events and context instead of giving minor visual details equal weight. Experiments across four video-captioning benchmarks found that Reviso outperformed the evaluated open-source baselines, and its captions better supported downstream video question answering.

Reviso begins with a low-frame-rate preview of a video, then builds a hierarchical Context–Focus–Points (CFP) plan. The plan identifies the overall context, prioritizes important aspects, and sets specific targets and tools for closer inspection. Reviso selectively revisits video segments, combines tools to examine them in detail, and updates its plan as it interprets the evidence. It then combines information from the full video and selected segments to produce a caption.

To teach this observation strategy, the researchers created a training pipeline that checks factual accuracy and refines examples with guidance from a teacher model. They trained Reviso with Group Relative Policy Optimization (GRPO), using a key-fact reward for caption quality, a Plan Alignment Reward (PAR) for selecting important observation targets, and an Unproductive Tool-Call Penalty (UTCP). UTCP adjusts penalties for observation costs based on caption quality. The paper reports results against evaluated open-source systems and says Reviso’s captions also improved support for video question answering; it gives no benchmark scores in the provided abstract.