Skip to content

论文HF Daily Papers · arXiv 2611.259803 天前

AFFECT训练目标提升世界模型动作响应保真度与机器人规划成功率 ​

本文由 AI 根据公开内容整理,可能有误,请以原文为准:Learning What Actions Change: Action Effect Matching for Dynamics-Faithful World Models(HF Daily Papers · arXiv 2611.25980)

核心内容 ​

现有世界模型虽能生成逼真视频,但常忽略微小动作扰动对未来观测的影响,限制了其在机器人策略评估与规划中的应用。研究团队提出AFFECT辅助训练目标,旨在让模型学习动作扰动引起的视觉变化分布。该方法通过在任务无关的人类遥操作数据中注入时间相关噪声,训练世界模型使其生成视频对噪声的响应与真实录制视频一致,无需从同一初始状态重复执行动作。

在OGBench仿真及DROID、ALOHA-2真机平台上验证,AFFECT显著提升了模型推演与真实结果的一致性。在DROID数据集上,世界模型内策略成功率与真机的Pearson相关系数由0.47提升至0.79,Fréchet Video Distance(FVD)指标最高降低23%。在下游规划任务中,采用经AFFECT训练的世界模型进行动作选择,使实际操作任务成功率最高提升8.3%,证实了其对动力学忠实度的改善效果。

关键看点 ​

  • AFFECT机制:向人类遥操作数据注入时间相关噪声,训练世界模型匹配真实视频对动作扰动的视觉响应分布
  • 动力学保真度:在DROID真机测试中,世界模型与实体机器人策略成功率的Pearson相关系数从0.47提升至0.79
  • 生成质量优化:引入AFFECT目标后,世界模型生成视频的FVD指标最高降低23%,视觉动态更贴近真实物理规律
  • 规划性能增益:基于AFFECT训练的世界模型进行动作选择,使机器人实际操作任务成功率最高提升8.3%

工程与产业观察 ​

AFFECT通过低成本的动作扰动匹配解决了世界模型“视觉逼真但动力学失真”的工程痛点。无需昂贵的重复采样数据即可显著提升Sim-to-Real相关性,为具身智能提供了更可靠的离线策略评估与Model-based Planning基座,降低了真机调试风险。

英文摘要 (Abstract) ​

World models aim to predict environment dynamics under actions, enabling the prediction of action consequences without executing them in the real world. Yet despite producing increasingly realistic videos, current world models often fail to capture how small action changes affect future observations. This limits their use in tasks such as robotic policy evaluation and model-based planning, where nearly identical actions can lead to qualitatively different outcomes. To address this, we propose AFFECT, an auxiliary training objective for learning the distribution of visual changes induced by action perturbations. Specifically, we inject temporally correlated noise into the actions of a policy trained on task-agnostic human teleoperation. We then train the world model so that its generated videos respond to this noise as the recorded videos do, without requiring repeated executions from the same initial state. Across OGBench simulation and real-world DROID and ALOHA-2 setups, AFFECT makes world-model rollouts more faithful to real outcomes. On DROID, the Pearson correlation between policy success rates in the world model and on the real robot rises from 0.47 to 0.79, alongside up to 23% lower FVD. These gains carry over to planning, where selecting actions with AFFECT-trained world models improves task success by up to 8.3%.

Hugging Face Daily Papers 热度:10 upvotes 作者:John Won, Yonghoon Dong, Jinwoo Shin

基于本页技术方向关联的整机参数、核心零部件、实测报告与同赛道选型横评。

外部来源

本页事实对应的公开页面,以原文为准。(点击将离开本站)

  1. HF Daily Papers · arXiv 2611.25980:Learning What Actions Change: Action Effect Matching for Dynamics-Faithful World Models ↗访问 2026-11-21