论文arXiv 2611.190475 天前
CAST框架:测试时认证筛选提升冻结扩散策略安全执行成功率
本文由 AI 根据公开内容整理,可能有误,请以原文为准:Certify, Then Select: Viability-Aware Runtime Steering of Frozen Diffusion Policies(arXiv 2611.19047)
核心内容
针对扩散策略部署后遭遇训练分布外约束且重训成本高的问题,研究提出CAST(Certified Action Selection at Test time)框架。该方法无需微调冻结策略,通过在测试时采样多个动作块,利用障碍条件进行安全认证,并由学习的价值集成对可行样本排序择优执行。理论证明认证步骤具备绝对安全性,且执行动作与可行性条件下的策略偏差控制在log N nats内,解决了传统安全滤波忽视任务目标的问题。
为解决单步认证无法保证长期可行性的死锁难题,进一步提出CAST-V与CAST-VL变体,通过有限样本保证或日志学习估计策略条件化生存能力(Viability)。在Robomimic操作、平面四旋翼、MuJoCo人形机器人及独轮车基准测试中,CAST在几乎所有设置下的联合任务与约束成功率均超越输出滤波器。实验表明,当滤波器编辑幅度较大时,“先认证后排序”机制至关重要,而引入生存能力评估消除了绝大多数单步认证不可见的死端状态。
关键看点
- 测试时认证机制:从冻结扩散策略采样多动作块,经障碍函数认证后由价值集成排序,确保执行动作既满足安全约束又保持任务相关性。
- 理论安全边界:证明认证步骤独立于Critic绝对安全,且可行样本执行偏差受控于log N nats,避免传统投影滤波导致的策略奇异性。
- 生存能力保障:CAST-V/VL通过模型估计或日志学习量化有限采样下的安全控制权限,解决生成式控制器当前安全但未来无解的死锁问题。
- 多平台验证:在Robomimic、MuJoCo人形机器人等四类基准中,联合成功率全面优于输出滤波器,证实先认证后排序在大编辑量场景的决定性作用。
工程与产业观察
CAST为具身智能扩散模型落地提供了关键的运行时安全兜底方案。其“认证-排序”范式解耦了策略学习与约束满足,使通用基础模型能在不重训前提下适配动态物理约束。引入的Viability概念将安全从瞬时状态扩展至轨迹可行性,对解决复杂非结构化环境中的长程任务死锁具有重要工程参考价值。
英文摘要 (Abstract)
Diffusion policies, which generate short action sequences by denoising random noise, are a leading way to learn robot skills from demonstrations. Once deployed, they meet constraints the demonstrations never contained, and retraining for each is impractical. A safety filter edits the policy's action to satisfy them, but the edit ignores the task, and a large edit can leave the policy where it cannot continue. We propose CAST (Certified Action Selection at Test time): draw a few action chunks from the frozen policy, certify each with a barrier condition, and execute the certified chunk that a learned value ensemble ranks highest. We prove that a certified step is safe whatever the critic does; that, whenever a certified sample exists, the executed chunk stays within log N nats of the policy conditioned on feasibility, while a safety-filter projection can be singular to it; and that an exact closed-loop identity splits the gain of ranking into availability, spread and the critic's target. Certifying the next chunk is not enough: a generative controller can have safe actions available now and none later. We characterise the safe control authority of a frozen policy under finite sampling, and CAST-V preserves it through a policy-conditioned viability, estimated with a model under a finite-sample guarantee or learned from the controller's own logs (CAST-VL). On robomimic manipulation, a planar quadrotor, a MuJoCo humanoid and a unicycle, with frozen policies, CAST raises joint task-and-constraint success above the output filter in nearly every setting; certifying before ranking is decisive where a filter's edit is large, and viability removes most dead ends that one-step certification cannot see. Project page: https://clmoro.github.io/cast.
作者:Angelo Moroncelli, Loris Roveda
外部来源
本页事实对应的公开页面,以原文为准。(点击将离开本站)