Skip to content

论文arXiv 2611.1034312 天前

GUARD框架:异常感知Q引导提升灵巧操作策略鲁棒性与真机迁移率 ​

本文由 AI 根据公开内容整理,可能有误,请以原文为准:GUARD: Anomaly-Informed Q-Steering for Generative Robot Policies in Dexterous Manipulation(arXiv 2611.10343)

核心内容 ​

针对扩散与流匹配等生成式机器人策略在闭环执行中易受协变量偏移影响的问题,研究提出GUARD异常感知Q引导框架。该方法无需人工干预或工程化奖励,直接利用策略训练分布构建密集故障预警信号。在四项Adroit灵巧操作基准测试中,GUARD在名义和扰动动力学下平均成功率分别达96.0%和80.0%,较最强基线分别提升21.0和20.0个百分点,同时保持最低异常分数,验证了其在复杂多模态行为建模中的有效性。

技术上,GUARD将稀疏任务奖励与从训练数据学习的密集异常评分结合,通过Q函数传播梯度以引导去噪过程,使动作优化任务回报的同时避开低支持度区域,且不修改预训练策略权重。该框架成功实现Sim-to-Real迁移,在真机手持螺丝刀旋转与重定向任务中,成功率较基线分别提升40和25个百分点。结果表明异常信号可作为稀疏奖励的有效补充监督,显著提升生成式策略在物理世界的部署可靠性。

关键看点 ​

  • 核心机制:结合稀疏任务奖励与密集异常评分,通过Q函数梯度引导去噪过程避开低支持度区域,无需微调预训练策略
  • 仿真性能:Adroit灵巧操作名义/扰动动力学成功率达96.0%/80.0%,较最强基线提升21.0/20.0个百分点且异常分最低
  • 真机迁移:Sim-to-Real直接迁移至手持螺丝刀旋转与重定向任务,成功率较基线分别提升40和25个百分点
  • 数据利用:仅依赖策略原始训练分布构建故障预警信号,摆脱对额外演示、人工干预或特定任务工程奖励的依赖

工程与产业观察 ​

GUARD证明了利用策略自身训练分布挖掘异常信号作为密集监督的有效性,解决了生成式策略Sim-to-Real落地中的协变量偏移痛点。该方案不侵入预训练模型,为灵巧操作提供了低成本安全对齐范式,对具身智能算法从仿真走向真实物理环境具有重要工程参考价值。

英文摘要 (Abstract) ​

Generative robot policies based on diffusion and flow matching can model complex, multimodal behaviors, but remain susceptible to covariate shift during closed-loop execution. Existing approaches mitigate such failures through additional demonstrations, policy fine-tuning, or test-time guidance, but often require human intervention, engineered rewards, or other task-specific signals. We introduce GUARD, an anomaly-informed Q-steering framework that leverages the policy's training distribution to construct a dense, failure-anticipatory steering signal. GUARD pairs sparse task rewards with dense anomaly scores learned from policy training data and propagates both through a learned Q-function. Its gradients then steer the denoising process toward actions that improve task return while preventing trajectories from entering poorly supported regions, without directly modifying the pretrained policy. Across four Adroit dexterous manipulation tasks, GUARD achieves average success rates of $96.0%$ and $80.0%$ under nominal and perturbed dynamics, outperforming the strongest baseline by $21.0$ and $20.0$ percentage points while maintaining the lowest anomaly score. GUARD further demonstrates direct sim-to-real transfer on in-hand screwdriver rotation and reorientation tasks, achieving $40$ and $25$ percentage-point gains in success rate. These results demonstrate that anomaly signals can serve not only for failure monitoring but also to complement sparse task rewards as effective supervision for robust policy steering.

作者:Aryaman Gupta, Xiaoyue Wu, Rana Soltani Zarrin, Mark Van der Merwe

基于本页技术方向关联的整机参数、核心零部件、实测报告与同赛道选型横评。

外部来源

本页事实对应的公开页面,以原文为准。(点击将离开本站)

  1. arXiv 2611.10343:GUARD: Anomaly-Informed Q-Steering for Generative Robot Policies in Dexterous Manipulation ↗访问 2026-11-11