Skip to content

论文HF Daily Papers · arXiv 2611.195598 天前

ResWAM残差世界动作模型:2.3B参数实现92.7ms低延迟具身控制 ​

本文由 AI 根据公开内容整理,可能有误,请以原文为准:ResWAM: Learning What Changes, Not What Stays, for World-Action Models(HF Daily Papers · arXiv 2611.19559)

核心内容 ​

针对现有世界动作模型(WAMs)因监督完整未来观测导致静态背景主导损失、计算开销大阻碍闭环部署的问题,研究提出ResWAM模型。该方法采用视觉残差学习策略,在冻结DINOv3特征空间中仅监督当前帧到未来帧的特征变化,使表征聚焦于机械臂与被操作物体而非背景。ResWAM总参数量2.3B,包含轻量级0.5B双分支DiT,通过条件流匹配联合训练动作与残差分支,显著降低算力需求。

在无外部机器人轨迹预训练条件下,ResWAM在LIBERO-Plus、RoboTwin 2.0及RoboDojo基准上分别取得81.1%、93.8%和9.40%的平均成功率,训练与推理算力仅需2.7-4.0 TFLOPs和1.0-1.4 TFLOPs。真机Open、Put、Stack任务平均成功率达80.0%,较π_{0.5}提升10个百分点,端到端推理延迟低至92.7ms,验证了残差预测机制在保持高性能的同时实现高效实时具身控制的工程可行性。

关键看点 ​

  • 残差学习机制:在冻结DINOv3空间仅监督帧间特征变化,提升特征秩并使表征集中于机械臂与物体运动,避免静态背景主导梯度更新
  • 轻量化双分支DiT:0.5B参数双分支架构通过条件流匹配联合训练动作与残差分支,总模型2.3B参数,推理算力降至1.0-1.4 TFLOPs
  • 真机低延迟部署:Open/Put/Stack任务平均成功率80.0%,较基线提升10%,端到端推理延迟92.7ms,满足高频闭环控制实时性要求
  • 零预训练SOTA性能:无需外部机器人数据预训练,在LIBERO-Plus与RoboTwin 2.0分别达81.1%与93.8%成功率,训练算力仅2.7-4.0 TFLOPs

工程与产业观察 ​

ResWAM以残差预测替代全量未来生成,将世界模型从视频生成范式转向任务相关动态建模,大幅压缩冗余计算。其92.7ms真机延迟与零预训练SOTA表现,为端侧部署VLA提供可复现路径,表明具身智能可从“大而全”转向“小而精”的残差表征学习。

英文摘要 (Abstract) ​

World-action models (WAMs) augment vision-language-action policies (VLAs) with future prediction so that the policy can anticipate how its actions will affect the physical world. Existing WAMs supervise complete future observations. However, observations typically change little over an action chunk, so the future loss is dominated by copying static background content rather than learning task-relevant robot and object motion, limiting policy learning. Moreover, existing WAMs rely on large video generation models or separate world models, incurring heavy training and inference compute that hinders closed-loop deployment. To improve policy performance, we propose visual residual learning, supervising only the feature change from the current frame to the future frame in a frozen DINOv3 space. Compared with complete future features, this residual learning target yields a higher feature rank with richer and more diverse representations of scene changes, and spatially concentrates on the robot arm and manipulated objects rather than the background. To further improve computational efficiency, we present ResWAM, a residual world-action model with 2.3B parameters in total, including a lightweight 0.5B dual-branch DiT that jointly trains an action branch and a residual branch via conditional flow matching. Without robot pretraining (i.e., pretraining on external robot action trajectories beyond the benchmark demonstrations), ResWAM achieves top average success rates on LIBERO-Plus (81.1%), RoboTwin 2.0 (93.8%), and RoboDojo (9.40% success, 13.41 score) at the lowest training (2.7-4.0 TFLOPs) and inference compute (1.0-1.4 TFLOPs), and leads real-robot Open, Put, and Stack (80.0% average, +10.0% over π_{0.5}) at 92.7 ms latency.

Hugging Face Daily Papers 热度:8 upvotes 作者:Chongyu Fan, Gaowen Liu, Ramana Rao Kompella, Jason Corso, Sijia Liu

基于本页技术方向关联的整机参数、核心零部件、实测报告与同赛道选型横评。

外部来源

本页事实对应的公开页面,以原文为准。(点击将离开本站)

  1. HF Daily Papers · arXiv 2611.19559:ResWAM: Learning What Changes, Not What Stays, for World-Action Models ↗访问 2026-11-18