论文arXiv 2611.251992 天前
VG-Steer:视觉几何策略引导使机器人电缆放置成功率翻倍
本文由 AI 根据公开内容整理,可能有误,请以原文为准:VG-Steer: Visual-Geometric Policy Steering from In-Context Aligned Abstractions(arXiv 2611.25199)
核心内容
针对通用机器人在分布外(OOD)设置下因数据稀缺导致端到端策略过拟合的问题,研究提出VG-Steer测试时自适应方法。不同于依赖隐式信号或粗粒度空间信息的现有方案,该方法通过迁移成功轨迹中的精细运动来修正冻结策略。仿真基准测试表明其能有效纠正流匹配策略的位姿与目标偏移;真机实验中,在电缆放置任务里,该算法将工作区覆盖范围扩大,使训练位置外一格的作业成功率提升超过一倍。
技术上,VG-Steer定义“物体-关系距离对”作为几何轴,结合大视觉模型(LVMs)与视觉语言模型(VLMs)的语义推理能力,生成定制化的视觉-几何引导信号。该信号融合了任务相关实体对及其距离投影,为策略提供精确修正依据。实验证实,该机制不仅提升了单一任务策略对未训练物体的泛化能力,还几乎完全消除了物体识别混淆现象,解决了纯视觉方法在微小分布漂移下的鲁棒性瓶颈。
关键看点
- 核心机制:利用LVM/VLM语义推理结合物体-关系距离对,构建视觉-几何引导信号修正冻结策略
- 真机性能:在电缆放置任务中,将训练分布外一格位置的操作成功率提升超过100%
- 泛化能力:引导单任务策略操作未训练物体,并几乎完全消除物体识别混淆现象
- 技术优势:克服传统测试时自适应方法依赖隐式信号及显式空间信息修正粒度过粗的缺陷
工程与产业观察
该工作验证了利用基础模型语义对齐几何抽象进行策略微调的有效性,为具身智能在少样本OOD场景下的落地提供了新思路。相比重训或复杂感知模块,这种基于视觉-几何信号的测试时引导方案工程开销低,显著提升了操作策略在非结构化环境中的鲁棒性与泛化边界。
英文摘要 (Abstract)
Adapting robot policies to out-of-distribution (OOD) settings with limited real-world data remains a key bottleneck to generalist robots. The multimodality and scarcity of robotics data lead end-to-end approaches to overfit and break under small distribution shifts. Test-time adaptation can alleviate this by correcting frozen policies at inference, but most methods ground their correction in a signal learned implicitly from robotic datasets, making it susceptible to the same shifts as the policy it corrects. Recent methods that instead exploit explicit spatial information are typically limited to coarse corrections, missing the nuanced, task-relevant motion most manipulation tasks require. We propose VG-Steer, which corrects existing policies by transferring the nuanced motion of successful in-distribution rollouts, captured along meaningful geometric axes, known here as object-relational distance pairs. VG-Steer leverages the semantic understanding of Large Vision Models (LVMs) and Vision-Language Models (VLMs) reasoning, which combines the task-relevant entity pairs and their object-relational distance projections into a single tailored visual-geometric guidance signal. In a simulation benchmark, it efficiently corrects flow-matching policies under pose and task-goal shifts; on a real robot experiment, it widens the region of the workspace the frozen policy covers, more than doubling its success rate on cable placements one grid step outside the training positions, and steers a single-task policy toward objects it was never trained on while nearly eliminating object confusion. Project website: https://arthurfenderbucker.github.io/VG-Steer/
作者:Arthur Bucker, Pramod Anantharam, Jonathan Francis, Jean Oh
外部来源
本页事实对应的公开页面,以原文为准。(点击将离开本站)