外观
论文arXiv 2610.1557217 天前
移动前先看：面向开放世界移动操作的智能体-接口对齐监督
原文链接：Look Before You Move: Agent-Interface-Aligned Supervision for Open-World Mobile Manipulation（arXiv 2610.15572 · 发布日期：2026-10-11）
Look Before You Move: Agent-Interface-Aligned Supervision for Open-World Mobile Manipulation
针对语言指令下的开放世界移动操作，提出AIDA数据对齐流程，并在OWMM、真机和导航子集上验证。
核心要点
- 开放世界移动操作需同时学习动作选择与主动视觉，最短路径进展难以监督看向何处。
- AIDA让教师通过部署智能体的技能接口直接选择并执行动作，收集接口对齐示范。
- 方法加入模板式空间推理监督，并用共享框架组织训练和推理中的观测、历史与输出。
- AIDA-OWMM在170个留出OWMM回合达71.76%成功，仿真训练后真机达38%。
验证范围与局限：验证范围包括170个留出OWMM仿真回合、一台真实移动机器人端到端测试，以及EmbodiedBench导航子集。
英文摘要 (Abstract)
Open-world mobile manipulation requires a robot to find an object, approach it, and move it to a target receptacle from a language instruction. Training vision-language model (VLM) agents for these tasks requires demonstrations of both action selection and active vision. Existing pipelines often convert planned trajectories into high-level action labels, but shortest-path progress provides limited supervision for deciding where to look. We introduce Agent-Interface Data Alignment (AIDA), a data collection pipeline in which a teacher directly selects and executes actions through the deployed agent's skill interface. For mobile manipulation, we design teachers that maintain target visibility, recover lost targets, and approach for picking or placing. We add template-based spatial reasoning supervision and use a shared harness to organize observations, history, and outputs during training and inference. The resulting agent, AIDA-OWMM, achieves 71.76% full-task success on 170 held-out OWMM episodes. Teacher sensitivity analysis shows that active-vision demonstrations improve student success. The simulation-trained agent achieves 38% end-to-end success on a real mobile robot without additional training. Applying AIDA to a navigation subset of EmbodiedBench further demonstrates the pipeline's utility for navigation tasks.
作者：Ronghe Qiu, Teli Ma, Jiaming Zhou, Zeying Gong, Yiting He, Qiang Huang, Junwei Liang
来源
本页事实对应的公开页面，以原文为准。