外观
论文arXiv 2409.010831 个月前
基于可供性并结合flow matching的机器人操作
原文链接：Affordance-based robot manipulation with flow matching（arXiv 2409.01083 · 发布日期：2026-09-22）
Affordance-based robot manipulation with flow matching
提出基于可供性与flow matching的辅助机器人操作框架，并在含10个任务的真实世界数据集上验证。
核心要点
- 面向辅助机器人操作，聚焦大模型场景可供性理解的高效适配与动作学习。
- 用冻结视觉模型加可学习文本提示预测可供性，并建模多任务空间与语义关系。
- 将视觉运动策略表示为随机路标流向目标动作的条件过程， grounding视觉可供性。
- 真实世界10任务实验显示，提示调优具参数效率，flow matching训练更稳、推理更快；代码已开源。
验证范围与局限：验证范围为包含10个任务的真实世界数据集。
英文摘要 (Abstract)
We present a framework for assistive robot manipulation that addresses two fundamental challenges: efficient adaptation of large-scale models for scene affordance understanding and effective learning of robot actions by grounding the visual affordance. To tackle the first challenge, we adopt a parameter-efficient prompt tuning method, prepending learnable text prompts to a frozen vision model to predict affordances, while considering spatial and semantic relationships in multi-task scenarios. For the second challenge, we propose a flow matching method, representing a robot visuomotor policy as a conditional process of flowing random waypoints to desired robot actions. We introduce a real-world dataset with 10 tasks to evaluate our approach. Experiments show our prompt tuning method achieves competitive or superior performance to other finetuning protocols across data scales, while satisfying parameter efficiency. Furthermore, flow matching yields more stable training and faster inference compared to diffusion policy; specifically, 1-step flow matching achieves comparable accuracy to 16-step DDIM while reducing inference time by roughly 85%. Our framework seamlessly unifies high-level parameter-efficient affordance representation learning with low-level flow matching policies, illustrating how explicit affordances serve as effective spatial grounding for flow-based policies. https://github.com/HRI-EU/flow_matching.
作者：Fan Zhang, Michael Gienger
来源
本页事实对应的公开页面，以原文为准。