外观
论文arXiv 2609.238631 个月前
GAM范式：3D接地提升机器人操作策略泛化与长程任务成功率
提出Grounded Action Model（GAM）范式，通过3D接地构建物体中心表征解决VLA隐式学习痛点。在RoboTwin 2.0及LIBERO-PRO刷新SOTA，真机双机械臂视觉偏移下成功率达85%，结合Molmo2规划器实现64.7%长程任务完成率。
Franka FR3本文由 AI 根据公开内容整理，可能有误，请以原文为准：Grounded Action Model: 3D Grounding as a Foundation for Robotics（arXiv 2609.23863）
核心内容
针对现有VLA与WAM模型缺乏显式度量接地导致操作策略泛化差的问题，研究提出Grounded Action Model（GAM）新范式。该模型支持语言、点或框提示输入，将其转化为共享的物体中心表征以捕获目标视觉特征与度量几何信息。在RoboTwin 2.0基准上，GAM于50项任务中取得55.3%平均成功率，场景随机化条件下达47.6%；在LIBERO-PRO的16种扰动设置中以61%成功率超越$π_{0.5}$基线，验证了3D接地对提升操作鲁棒性的有效性。
架构层面，GAM利用多流Transformer融合物体中心表征与机器人状态历史来预测动作块，既可自主运行也可作为底层控制器对接高层规划器。真机实验中，双臂YAM机器人在视觉偏移下保持17/20成功率，显著优于$π_{0.5}$的4/20；搭配Molmo2规划器的Franka机器人在长程及记忆依赖任务中实现64.7% ID与49.8% OOD步骤完成率。这表明显式3D接地能有效弥补纯数据驱动策略在分布外场景下的短板，为具身智能提供可靠底层控制接口。
关键看点
- 3D接地范式：将语言/点/框提示转为物体中心表征，融合度量几何与视觉特征，解决VLA隐式学习缺陷
- 仿真性能SOTA：RoboTwin 2.0随机场景成功率47.6%，LIBERO-PRO扰动测试达61%，均超越现有基线模型
- 真机抗干扰性：双臂YAM视觉偏移下成功率85%，远超对比模型20%，验证显式几何表征的Sim-to-Real价值
- 长程任务组合：GAM作底层控制器配合Molmo2规划器，Franka真机ID/OOD步骤完成率分别达64.7%和49.8%
工程与产业观察
GAM证明了在具身基础模型中引入显式3D接地比单纯扩大VLA训练数据更能提升操作鲁棒性。其作为通用底层控制器的架构设计，解耦了感知接地与高层规划，为长程复杂任务提供了模块化解决方案。这对解决当前具身智能在非标环境下的泛化瓶颈具有直接工程参考价值，提示行业应重视几何先验在端到端策略中的锚点作用。
英文摘要 (Abstract)
Manipulation policies must know which objects matter and where they are, yet the pretrained backbones that current robot foundation models build on, from language in vision-language-action models (VLAs) to video generation in world-action models (WAMs), do not directly require this metric grounding, leaving it to be learned implicitly from robot demonstrations. We propose Grounded Action Models (GAMs), a new paradigm of robot foundation models built with 3D grounding. GAM can be conditioned using language, points, or box prompts, which are first transformed into a shared object-centric representation of the selected objects. This representation captures target-focused visual features and metric object geometry, which is mixed with robot state history through a multi-stream transformer to predict action chunks. Although GAMs can be run autonomously, they can also serve as a low-level controller that a high-level planner controls using its various input modalities, allowing for long-horizon and memory-dependent manipulation. On RoboTwin 2.0, GAM achieves an average success rate of 55.3% across 50 tasks (vs. 52.0% for Spatial Forcing), including 47.6% under scene randomization (vs. 30.4% for Abot-M0), with its action policy trained only on clean-scene demonstrations. On LIBERO-PRO, it achieves a state-of-the-art average success rate of 61% (vs. 53% for $π_{0.5}$) across 16 perturbation settings, with the largest gains when targets are relocated or newly designated. On two real robots, GAM retains 17/20 successes under visual shift on a bimanual YAM versus 4/20 for $π_{0.5}$, while its composition with a Molmo2 planner on a Franka achieves 64.7% ID and 49.8% OOD step completion on long-horizon and memory-dependent tasks.
作者：Gehao Zhang, Weikai Huang, Shailesh Shailesh, Yiyan Peng, Jiafei Duan, Ranjay Krishna
来源
本页事实对应的公开页面，以原文为准。