具身开发者社区

外观

论文arXiv 2610.1982514 天前

Vision-Language-Action 模型的世界建模中什么最重要？ ​

研究VLA中世界模型集成的关键因素，统一比较四类范式、多种表征和骨干，并在多种设置含真实世界中验证。

AI 整理
  • 感知与导航
  • 数据与采集
  • 具身模型

原文链接：What Matters for World Modeling in Vision-Language-Action Models?（arXiv 2610.19825 · 发布日期：2026-10-14）

What Matters for World Modeling in Vision-Language-Action Models?

研究VLA中世界模型集成的关键因素，统一比较四类范式、多种表征和骨干，并在多种设置含真实世界中验证。

核心要点 ​

  • 动机是现有VLA世界模型方法在表征、集成范式和策略骨干上同时变化，难以判断关键因素。
  • 将相关方法归为Implicit Future Alignment、Explicit Future Query、MoT Future Expert和External World Predictor四类，并在统一框架中实例化。
  • 系统评估多种世界表征和策略骨干，发现2D语义表征具有跨范式收益，MoT Future Expert整体竞争力强且OOD稳健。
  • 世界建模在多种设置中优于仅动作基线，长程和动态任务收益明显；项目页和视频演示已提供。

验证范围与局限：验证范围包括域内、域外、长程、动态、少样本和真实世界设置，摘要未说明具体任务或平台。

英文摘要 (Abstract) ​

Integrating world models into Vision-Language-Action models (VLAs) has emerged as a promising direction, enabling policies to anticipate future dynamics beyond reactive observation-to-action mapping. However, existing approaches vary simultaneously in their world representations, integration paradigms, and policy backbones, making it difficult to determine what truly matters for effective world model integration. To disentangle these factors, we categorize existing efforts into four representative paradigms, namely Implicit Future Alignment, Explicit Future Query, MoT Future Expert, and External World Predictor. We instantiate each within a unified framework and systematically evaluate these paradigms with diverse world representations and policy backbones across in-domain, out-of-domain, long-horizon, dynamic, few-shot, and real-world settings. Our study yields four main findings: 1) world representation effectiveness is paradigm-dependent, whereas 2D semantic representations offer robust, cross-paradigm gains; 2) no single paradigm is universally optimal, yet the decoupled MoT Future Expert offers competitive overall performance, particularly in OOD robustness; 3) world modeling consistently benefits policies built on standard VLMs, while decoupled designs exhibit greater compatibility with pretrained VLAs; and 4) world modeling improves performance over action-only baselines across multiple settings, with notable gains on long-horizon and dynamic tasks. Project page and video demonstrations are available at https://dylanmqs.github.io/WM-in-VLAs/.

作者：Qisen Ma, Yixiang Chen, Yuan Xu, Peiyan Li, Jiabing Yang, Kai Wang, Jianhua Yang, Jianlou Si, Jun Huang, Jing Liu

来源

本页事实对应的公开页面，以原文为准。

  1. arXiv 2610.19825：What Matters for World Modeling in Vision-Language-Action Models?访问 2026-10-21