外观
论文arXiv 2609.282961 个月前
Talk2Escape：用于视觉-语言导航的对话式落地
为解决VLN开环易累积偏差失效，Talk2Escape以监测运动触发对话纠错，在仿真与真机上验证。
宇树 Go2原文链接：Talk2Escape: Conversational Grounding for Vision-and-Language Navigation（arXiv 2609.28296 · 发布日期：2026-09-23）
Talk2Escape: Conversational Grounding for Vision-and-Language Navigation
为解决VLN开环易累积偏差失效，Talk2Escape以监测运动触发对话纠错，在仿真与真机上验证。
核心要点
- 单轮VLN让智能体开环执行，感知混淆、噪声与里程计漂移会累积并导致失败。
- 该模型无关框架用轻量视觉语言模块持续监测运动，发现循环或严重偏离即介入。
- 系统将自我中心观测转为简洁落地的问题，向算法oracle或人类获取定向纠正反馈。
- 在R2R-CE、RxR-CE、VLNVerse中跨多种基础智能体持续提升，R2R-CE成功率达66.0%。
验证范围与局限：验证范围包括高保真仿真器R2R-CE、RxR-CE、VLNVerse，以及Unitree Go2四足机器人的sim-to-real迁移。
英文摘要 (Abstract)
While Vision-and-Language Navigation (VLN) has demonstrated remarkable success, the prevailing single-turn paradigm exposes a fundamental vulnerability: agents operate in a strictly open-loop manner. In practice, factors such as perceptual aliasing, sensor noise, and odometry drift can cause minor deviations to accumulate over time, often leading to catastrophic mission failures with no built-in mechanism for error recovery. To address this, we introduce \textit{Talk2Escape}, a proactive and model-agnostic dialogue intervention framework that reframes navigation as a closed-loop interactive process. At its core, a lightweight vision-language module continuously monitors agent kinematics. Upon detecting localized looping or severe trajectory divergence, it translates raw egocentric observations into concise, grounded queries to solicit targeted corrective feedback from either an algorithmic oracle or a human-in-the-loop. Extensive evaluations in high-fidelity simulators, including R2R-CE, RxR-CE, and VLNVerse, demonstrate that \textit{Talk2Escape} exhibits consistent improvements across diverse base agents. Empirically, \textit{Talk2Escape} achieves a 66.0% Success Rate on R2R-CE, outperforming the current supervised and zero-shot state-of-the-art methods. We further validate its sim-to-real transfer on a Unitree Go2 quadruped, proving that proactive dialogue drastically improves navigation robustness in physical environments.
作者：Zerui Li, Sihao Lin, Yanyan Shao, Jiwen Zhang, Xiangyu Shi, Shijie Li, Qi Wu
来源
本页事实对应的公开页面，以原文为准。