StreamDuo Brings Wiki Research to Game Commentary

Researchers introduced StreamDuo, a two-agent system for live commentary on games it has not encountered during training. The paper, published on October 31, pairs a trainable commentator with a frozen research agent that retrieves relevant entries from the game’s wiki. When the commentator sees an event it cannot identify, it sends the current video frames to the research agent; the agent returns several wiki entries two seconds later while commentary continues.

The researchers trained the commentator only on Baldur’s Gate 3, using two stages. Information-gain labels teach it to ask for wiki information only when an answer would help it describe what a human caster said. On-policy self-distillation (training from a model’s own outputs) teaches it to use retrieved answers even when they are noisy. The research agent remains frozen, and the system does not train on the target games.

Tested on 10 unseen games, StreamDuo raised entity recall from 0.082 to 0.116 compared with Proact-VL retrained on the same data. It reduced source-game leakage, or carryover of knowledge about the training game, from 9.9% to 1.0%. The paper also reports that StreamDuo achieved the best timing and highest judged liveliness and overall quality among the streaming models evaluated; it gives no numerical scores for those judgments in the abstract. The paper lists Zixian Gao, Penghao Luo, Xiangcheng Cao, Lixin Xiu, Ziqi Cai, Zhen Li, Erwin Wu and Kaipeng Zhang as authors.

Researchers introduced StreamDuo, a two-agent system for live commentary on games it has not encountered during training. The paper, published on October 31, pairs a trainable commentator with a frozen research agent that retrieves relevant entries from the game’s wiki. When the commentator sees an event it cannot identify, it sends the current video frames to the research agent; the agent returns several wiki entries two seconds later while commentary continues.

The researchers trained the commentator only on Baldur’s Gate 3, using two stages. Information-gain labels teach it to ask for wiki information only when an answer would help it describe what a human caster said. On-policy self-distillation (training from a model’s own outputs) teaches it to use retrieved answers even when they are noisy. The research agent remains frozen, and the system does not train on the target games.

Tested on 10 unseen games, StreamDuo raised entity recall from 0.082 to 0.116 compared with Proact-VL retrained on the same data. It reduced source-game leakage, or carryover of knowledge about the training game, from 9.9% to 1.0%. The paper also reports that StreamDuo achieved the best timing and highest judged liveliness and overall quality among the streaming models evaluated; it gives no numerical scores for those judgments in the abstract. The paper lists Zixian Gao, Penghao Luo, Xiangcheng Cao, Lixin Xiu, Ziqi Cai, Zhen Li, Erwin Wu and Kaipeng Zhang as authors.