论文HF Daily Papers · arXiv 2610.407484 天前

WorldBridge模型融合视频与动作专家，操作任务泛化成功率达96%

研究提出WorldBridge世界动作模型，通过结构化注意力连接视频与动作专家并采用联合流匹配训练。在RoboTwin 2.0全数据设置下成功率达96.23%，验证了视频预测与稀疏动作监督对具身任务泛化的互补增强效应。

AI 整理

仿真与训练
数据与采集
具身模型

本文由 AI 根据公开内容整理，可能有误，请以原文为准：WorldBridge: Bridging Video and Action for Task Generalization in World Action Models（HF Daily Papers · arXiv 2610.40748）

核心内容

针对具身模型物理环境任务泛化难题，研究团队提出WorldBridge世界动作模型（WAM）。实验表明，仅使用分布内视频-动作监督时，RoboTwin 2.0分布外任务成功率仅为25.07%；引入无标签视频后升至55.90%，每个任务增加1条动作标注示范即提升至70.70%。该发现证实广泛视频覆盖可激活逆动力学能力，而稀疏动作监督能进一步强化迁移效果。

WorldBridge架构包含独立的视频与动作专家模块，通过结构化注意力机制交互，并采用联合流匹配进行端到端训练。系统集成双时间尺度视觉记忆与动作强制机制，配套统一数据处理管线以标准化异构机器人轨迹及第一人称手部监督数据。在多基准测试中，该模型在RoboTwin 2.0全数据、Clean-to-Random、RoboCasa GR1 TableTop及DOMINO Level-1上分别取得96.23%、80.9%、82.8%和57.6%的成功率，达到当前最优水平。

关键看点

泛化增益量化：RoboTwin 2.0分布外任务中，无标签视频使成功率从25.07%升至55.90%，单条动作示范进一步提升至70.70%
核心架构设计：采用独立视频/动作专家加结构化注意力机制，结合联合流匹配训练与双时间尺度视觉记忆模块
多基准SOTA性能：RoboTwin 2.0全数据成功率96.23%，RoboCasa GR1 TableTop达82.8%，DOMINO Level-1为57.6%
异构数据管线：构建统一处理流程，标准化过滤异构机器人轨迹数据及第一人称手部操作衍生监督信号

工程与产业观察

该研究量化了视频预训练与稀疏动作微调在具身泛化中的边际收益，为降低真机数据采集成本提供实证依据。其分离式专家架构与联合流匹配范式，解决了多模态对齐难题，对构建通用操作基础模型及高效利用互联网视频数据具有重要工程参考价值。

英文摘要 (Abstract)

Achieving the broad task generalization of embodied models in physical environments remains a central challenge in robot learning. We study this challenge in World Action Models (WAMs) through two complementary capabilities: video generalization, which predicts the visual evolution of new tasks, and action generalization, which translates these predictions into executable behavior through a video-to-action bridge. By varying video and action supervision for tasks outside an initial training set, we examine their respective contributions to task transfer. On RoboTwin 2.0, using only in-distribution video-action supervision yields a 25.07% success rate on out-of-distribution tasks. Introducing videos of these tasks without their action labels raises success to 55.90%, while adding just one action-labeled demonstration per task further improves it to 70.70%. These findings suggest that broader video coverage enables task transfer by leveraging existing inverse-dynamics capabilities, while even sparse action supervision further strengthens those capabilities. Guided by this perspective, we introduce WorldBridge, a WAM with separate video and action experts connected through structured attention and trained with joint flow matching. WorldBridge incorporates visual memory at two temporal scales and action forcing, alongside a unified data processing pipeline that standardizes and filters heterogeneous robot trajectories and egocentric hand-derived supervision. Across multiple manipulation benchmarks, WorldBridge achieves state-of-the-art performance among the compared methods, including 96.23% on RoboTwin 2.0 full data setting and 80.9% on clean to random setting, 82.8% on RoboCasa GR1 TableTop, and 57.6% on DOMINO Level-1. Our study highlights the complementary roles of video prediction and action grounding in developing robots that generalize across tasks.

Hugging Face Daily Papers 热度：12 upvotes 作者：Jun Cen, Yubo Guo, Hao Li, Liangkai Li, Yaoxu Lyu, Meng Wei, Zeqi Xiao, Sizhe Yang, Yuqiang Yang, Yifan Ye, Shanghang Zhang, Duo Zheng