外观
论文HF Daily Papers · arXiv 2610.1936114 天前
VLX-Go多阶段学习实现58%干扰环境下具身跟踪成功率
本文由 AI 根据公开内容整理，可能有误，请以原文为准：VLX-Go: Robust Embodied Tracking through Multi-Stage Learning and On-Policy Refinement（HF Daily Papers · arXiv 2610.19361）
核心内容
针对共享空间中机器人跟随目标易受干扰物影响的问题，研究团队提出VLX-Go视觉语言航点策略。该方法采用多阶段训练范式，在19532个Oracle回合数据上完成从视觉语言对齐、多模态推理到跟踪专精的渐进式学习。实验显示，其干扰环境跟踪成功率达55.80%，较前代OmTrackVLA模型的41.54%显著提升，确立了分阶段多模态学习在复杂跟随任务中的有效性。
系统架构融合冻结视觉编码器、多尺度时序上下文与Transformer骨干网络，输出以机器人为中心的航点预测。进一步引入基于过滤Oracle标签的在策略优化机制，利用策略实际访问状态进行微调，使干扰跟踪成功率再增2.42个百分点至58.22%，同时碰撞率下降1.99个百分点。该方案在干扰及歧义场景下均取得对比方法中最优性能，项目代码已在GitHub开源。
关键看点
- 多阶段训练：依次执行视觉语言对齐、多模态推理与跟踪专精，基于19532个Oracle回合数据迭代优化策略
- 在策略优化：利用策略访问状态的过滤Oracle标签进行微调，干扰跟踪成功率提升2.42%至58.22%
- 安全性能提升：通过在策略优化机制，机器人在复杂跟随任务中的碰撞率降低1.99个百分点
- 模型架构：结合冻结视觉编码器、多尺度时序上下文与Transformer，输出机器人中心航点预测
工程与产业观察
VLX-Go验证了“先对齐后控制”的分阶段训练对解决具身跟踪中语义混淆问题的关键作用。其基于真实访问状态的在策略优化思路，为缓解仿真到真机迁移中的分布偏移提供了低成本路径。开源代码与明确的多模态学习范式，为人形机器人动态跟随算法的工程化落地提供了可复现的高性能基线。
英文摘要 (Abstract)
Embodied visual tracking requires a robot to identify a designated target from visual observations and control its own motion to follow that target through an environment. Following an isolated person is a comparatively simple demonstration of embodied visual tracking, but following the intended person among distractors remains challenging. Existing methods exhibit a substantial drop in success from single-target to distractor tracking, motivating a focus on the shared spaces that person-following robots must navigate. We present VLX-GO, a vision-language waypoint policy built around the hypothesis that visual and language representations should be aligned before tracking-specific motion is learned. The policy combines complementary frozen visual encoders, multiscale temporal context, a transformer backbone, and a lightweight head for robot-centric waypoint prediction. Its training progresses through vision--language alignment, multimodal reasoning mid-training, and tracking specialization on 19,532 oracle episodes. Compared with 41.54% distractor-tracking success for our earlier OmTrackVLA model, the complete multi-stage recipe achieves 55.80%. We then explore on-policy refinement using filtered oracle labels for states visited by the policy. This preliminary study adds 2.42 percentage points of distractor-tracking success, reaching 58.22%, while reducing collision rate by 1.99 pts. The final policy records the highest distractor- and ambiguous-tracking success rates among the compared methods. These results support staged multimodal learning as a promising direction for tracking among distractors and motivate further study of on-policy refinement for person following in shared spaces. Code is available at https://github.com/om-ai-lab/VLX-Go.
Hugging Face Daily Papers 热度：1 upvotes 作者：Kyusong Lee, Heting Ying, Tiancheng Zhao
来源
本页事实对应的公开页面，以原文为准。