外观
论文arXiv 2610.1027021 天前
Video Prediction Policy 2：预测更好，行动更好
原文链接：Video Prediction Policy 2: Predict Better, Act Better（arXiv 2610.10270 · 发布日期：2026-10-07）
Video Prediction Policy 2: Predict Better, Act Better
VPP2面向开放式操作中WAM预测错误问题，通过操作视频预训练、蒸馏与MoT动作模块，在视频预测、真机和基准任务上验证。
核心要点
- 现有WAM在开放环境常产生错误运动预测并导致动作错误，限制来自视频模型不适配操作及动作融入损害泛化。
- VPP2用大规模多样操作视频继续预训练基础视频模型，结合详细标注进行事件级视频预训练。
- 该方法将视频模型后训练并蒸馏为固定预测时域的单步视觉规划器，再以MoT架构学习隐式逆动力学。
- 实验显示VPP2在开放式视频预测、真实零样本ALOHA任务及LIBERO/RoboDojo基准上优于对比方法。
验证范围与局限：验证范围包括开放式任务视频预测、真实零样本ALOHA操作任务，以及经基准特定后训练的LIBERO-Pro、LIBERO-OOD和RoboDojo。
英文摘要 (Abstract)
World action models (WAMs) have emerged as an important class of generalist robot policies, aiming to transfer video prediction priors to action learning. However, we find that existing WAMs frequently produce incorrect motion predictions in open-ended environment, leading to erroneous actions. We attribute this limitation to two factors: (1) base video models are not optimized for manipulation, and (2) naively incorporating action components into video models can substantially degrade their generalization capabilities. We introduce Video Prediction Policy 2 (VPP2), a WAM that enables strong zero-shot generalization in both video prediction and action generation. First, we curate a large-scale, diverse dataset of manipulation videos to continue pretraining the base video foundation model. We annotate video clips with detailed captions and perform \textit{event-level} video pretraining to promote generalization across open-ended manipulation tasks. Second, we post-train and distill the video model into a single-step visual planner with fixed prediction horizon. Finally, we introduce action module via a mixture-of-transformers (MoT) architecture to learn implicit inverse dynamics model. Experiments demonstrate three key results: (1) VPP2-14B outperforms Cosmos3-64B by 11.0% points in video prediction instruction-following success rate on open-ended tasks; (2) VPP2 surpasses the strongest baseline by 18.5% points in success rate on real-world zero-shot ALOHA manipulation tasks; and (3) following benchmark-specific post-training, VPP2 achieves the highest success rates among evaluated methods on the challenging LIBERO-Pro, LIBERO-OOD, and RoboDojo benchmarks.
作者：Yanjiang Guo, Haodong Yan, Zhide Zhong, Zhongru Zhang, Qingyuan Yang, Qingzhou Lu, Xiaoyu Chen, Yen‐Jen Wang, Shuying Deng, Chenghan Yang
来源
本页事实对应的公开页面，以原文为准。