外观
论文HF Daily Papers · arXiv 2610.301367 天前
AW-0框架融合代码规划与世界模型提升双臂操作性能
本文由 AI 根据公开内容整理，可能有误，请以原文为准：Agentic World Action Models（HF Daily Papers · arXiv 2610.30136）
核心内容
针对现有世界动作模型缺乏显式技能复用机制、Agentic机器人缺乏物理动力学预测的问题，研究提出AW-0框架。该方案将代码规划与联合视频动作预测集成，在RoboTwin 2.0基准及真实物理双臂操作任务评测中，展现出超越传统VLA和WAM基线的任务执行性能，并在语言遵循、语义推理及系统鲁棒性方面取得显著提升，验证了代码作为规划与执行连接纽带的有效性。
技术上，AW-0基于统一Mixture-of-Transformers架构，将代码、未来视频和机器人动作建模为三个耦合生成流。Code Expert生成参数化技能例程与控制逻辑以结构化多步预测，Video与Action Expert分别生成视觉状态与底层电机指令。配套自动化数据管线可对机器人及人类视频轨迹进行程序对齐标注，并将重复例程整合为可复用技能库，实现规划、预测与执行的端到端打通。
关键看点
- 混合架构：采用Mixture-of-Transformers联合建模代码、视频与动作三股生成流，实现规划与物理预测耦合
- 代码专家模块：Code Expert生成参数化技能例程与控制逻辑，将技能表征为可复用程序以支持Agentic执行
- 自动化数据管线：自动标注人机视频轨迹并对齐程序，将高频例程 consolidation 为标准化可复用技能库
- 实测性能验证：在RoboTwin 2.0及物理双臂操作中，任务表现、语义推理与鲁棒性全面优于VLA/WAM基线
工程与产业观察
AW-0通过引入结构化代码流解决了世界模型“能预测但难复用”的工程痛点。其自动化技能库构建管线降低了具身数据清洗门槛，而代码作为中间表征增强了长程任务的可解释性与泛化能力，为双臂操作等复杂场景提供了兼顾物理接地与逻辑推理的新范式。
英文摘要 (Abstract)
Recent world action models provide visual foresight and future prediction for robot manipulation, but lack explicit mechanisms for compositional skill reuse. Agentic robotics combines code-based planning and reasoning with reusable skills, but often lacks predictive grounding in physical dynamics. We introduce AW-0, an agentic world action model framework that integrates code-based planning with joint video and action prediction. We jointly model code, future video, and robot actions as three coupled generative streams within a unified mixture-of-transformers architecture. Within this architecture a code expert generates parameterized skill routines and control logic while video and action experts generate future visual states and low-level motor commands. Code serves two complementary roles: it structures multi-step prediction and control within the world model and represents skills as reusable programs for subsequent agentic execution. We further introduce an automated curation pipeline that labels robot and human video trajectories with aligned programs and consolidates recurring routines into reusable skill libraries. Evaluations on RoboTwin 2.0 and physical bimanual manipulation tasks show improved task performance over VLA and WAM baselines, alongside gains in language following, semantic reasoning and robustness, demonstrating that agentic code enables connection between planning, prediction and execution within WAMs.
Hugging Face Daily Papers 热度：5 upvotes 作者：Zhuoyang Liu, Dantong Niu, Botong Zhang, Boning Shao, Jing Wang, Ruijie Zheng, Yuvan Sharma, Runyu Lu, Jiaming Liu, Renrui Zhang, Roei Herzig, Yutong Bai, Ken Goldberg, Danfei Xu, Yuke Zhu, Guanzhi Wa
来源
本页事实对应的公开页面，以原文为准。