论文HF Daily Papers · arXiv 2611.0111222 天前
Auriga框架弥合VLM规划执行鸿沟,零样本操作成功率达57.7%
本文由 AI 根据公开内容整理,可能有误,请以原文为准:Auriga Technical Report: A Task-Agnostic Agent Harness for Zero-Shot Embodied Manipulation(HF Daily Papers · arXiv 2611.01112)
核心内容
针对视觉语言模型在机器人操作中规划与执行脱节的问题,研究团队提出任务无关智能体框架Auriga。该框架在RoboDojo基准测试中实现57.7%的零样本操作成功率(得分64.0),较参考控制接口提升逾一倍,并在场景随机化条件下保持榜首,而同期学习策略性能下降达33.9%。在RoboCasa365复合任务中,同一冻结参数框架取得73.2%成功率,超越VLA辅助智能体最高23.1%。
技术上,Auriga构建显式任务信念状态,支持直接工具调用或多步Python程序合成;底层提供校准3D测量、手指参考系双臂控制及统一准入网关。执行前模型可在暂停仿真中预览视觉轨迹,运行时框架实时校验阶段容差与抓取边界,检测到偏差即停机并返回量化误差供原位恢复。该方案经13轮跨任务能力迭代后冻结评测,有效弥合了高层语义决策与物理执行间68.3%的性能缺口。
关键看点
- 性能跃升:RoboDojo零样本成功率从22.5%升至57.7%,缩小68.3%规划执行差距
- 安全预检:执行前暂停仿真预览轨迹,运行时校验容差并反馈量化误差供原位恢复
- 泛化验证:RoboCasa365复合任务成功率73.2%,超VLA辅助智能体最高23.1%
- 开源资产:完整代码与项目页已公开,支持任务无关双臂操作快速复现与部署
工程与产业观察
Auriga验证了通过结构化中间层而非端到端微调来弥合VLM语义理解与物理执行鸿沟的有效性。其仿真预检与运行时容差反馈机制为具身智能提供了可工程化的安全兜底范式,对降低真机试错成本、提升零样本部署可靠性具有直接参考价值,尤其适用于双臂协作等高精度操作场景。
英文摘要 (Abstract)
Vision-language models can interpret instructions and plan manipulation tasks, yet struggle when acting directly as robot policies. On RoboDojo, a large planning-execution gap separates high-level task decisions from physical execution: removing arm execution raises GPT-6 Astra's success rate from 22.5% under the reference control interface to 76.2%. We present Auriga, a task-agnostic agent harness that closes 68.3% of this gap in zero-shot manipulation without target-task trials or resets. In Auriga, the model makes all task decisions over an explicit task belief and acts through either direct tool calls or synthesized multi-step Python programs, while the harness supplies calibrated 3D measurements, finger-referenced dual-arm control, and a single admission gateway to the robot. Before motion, the model inspects a visual trajectory preview while simulation is paused; during execution, the harness checks declared stage tolerances and grasp bounds, stopping the robot on divergence and returning the measured discrepancy for in-place recovery. Developed across 13 cross-task capability iterations and frozen for evaluation, Auriga ranks first on the RoboDojo leaderboard with a 57.7% success rate (64.0 Score), more than doubling the reference harness on the same model, and remains first under scene randomization while learned policies drop by up to 33.9%. On RoboCasa365, the same frozen harness reaches a 73.2% success rate, outperforming VLA-assisted agents by up to 23.1% on composite tasks. Project page: https://auriga.stelledge.com/ Code: https://github.com/lose4578/Auriga
Hugging Face Daily Papers 热度:4 upvotes 作者:Chengcheng Wang, Yuheng Shi, Siyu Xu, Yutong Wang, Jinjing Zhao, Chenghao Xia, Jinqi Liao, Jiankun Peng, Ligong Bi, Yuechuan Ma, Jiaqi Liu, Wenbo Wang, Zijian Wang, Yuemin Wu, Minjing Dong, Xiu Su, Ta…
外部来源
本页事实对应的公开页面,以原文为准。(点击将离开本站)