论文arXiv 2611.04080今天

FEIJOA框架实现Spot移动操作94.2%语义锚定与70%任务完成率

针对移动操作语义理解难题，FEIJOA框架通过测试时动态查询RGB-D扫描实现94.2%指令锚定准确率。在Spot机器人50次实测中完成35项任务，仅2次失败源于推理，验证了具身感知闭环对长尾任务的有效性。

AI 整理

感知与导航
具身模型

本文由 AI 根据公开内容整理，可能有误，请以原文为准：FEIJOA: Task-Driven Grounding and Execution for Mobile Manipulation（arXiv 2611.04080）

核心内容

针对移动机器人在开放场景中因预建地图缺乏任务相关属性而导致执行失败的问题，研究提出FEIJOA语言模型智能体框架。该系统在测试阶段直接基于RGB-D扫描进行动态语义锚定，而非依赖离线构建的场景表示。在103个开放式锚定查询测试中，FEIJOA实现了94.2%的精确解析率，显著优于最佳预建表示方法的68.0%，解决了“收拾玩具”等模糊指令下的物体识别与容器匹配难题。

系统集成了具身感知校验机制，在执行前检查技能前置条件并采样可达基座姿态，执行后通过观测验证效果。在Boston Dynamics Spot机器人的50次真实世界试验中，成功完成35项任务，其中17次得益于工具调用或验证失败后的自动恢复机制。在15次失败案例中，仅2次归因于锚定或推理错误，其余13次均为物理技能限制，表明该架构已将语义理解瓶颈转移至底层运动控制层面。

关键看点

动态语义锚定：基于RGB-D扫描实时查询并迭代优化候选对象，在103个开放查询中达94.2%精确解析率
具身执行校验：集成前置条件检查、可达基座采样及执行前后观测验证，支持从工具调用失败中自动恢复
实测性能归因：Spot机器人50次实测完成35项任务，失败案例中87%源于物理技能而非语义推理缺陷
对比基准提升：相比最佳预建场景表示方法，指令解析准确率提升26.2个百分点，验证测试时接地优势

工程与产业观察

FEIJOA证明了在移动操作中，测试时动态感知比静态预建地图更能应对开放词汇指令。其将语义推理与物理执行解耦的架构设计，使算法工程师能独立优化高层规划与底层控制。高达94.2%的锚定准确率表明，结合VLM与几何验证是解决具身智能长尾语义问题的有效工程路径。

英文摘要 (Abstract)

A mobile manipulator asked to "put the toys away in the toy box" must decide which objects are toys and which container the user means, then find base poses, grasps, and placements it can execute. Because a scene representation is built before the instruction is known, it may lack the attributes, affordances, or concepts needed to resolve the task. We present FEIJOA, a language-model agent that grounds the instruction at test time. It queries a grounding tool over a posed RGB-D scan, inspects the returned candidates, and refines its queries until they resolve the instruction. An embodiment-aware harness checks each skill's preconditions, samples base poses from which the target is reachable, and verifies the skill's effect from observations before and after execution. On 103 open-ended grounding queries, FEIJOA resolves 94.2% exactly, compared with 68.0% for the best prebuilt representation. Over 50 real-world trials on Spot, it completes 35 tasks; 17 of these successes follow recovery from a failed tool call or verification check. Of the 15 failed trials, 13 were caused by the robot's physical skills, and only 2 by grounding or the agent's reasoning. Project website: https://williamshen-nz.github.io/feijoa/.

作者：Aditya Agarwal, William Shen, Sahit Chintalapudi, Lucy Cai, Nishanth Kumar, Aidan Curtis, Travis Manderson, Tom'as Lozano-P'erez, Leslie Pack Kaelbling