论文arXiv 2611.0865413 天前
CS-Pi推理期约束采样:Franka零样本安全成功率提升至55%
本文由 AI 根据公开内容整理,可能有误,请以原文为准:CS-Pi: Constraint-Satisfying Flow Policy Sampling(arXiv 2611.08654)
核心内容
针对流匹配策略在部署阶段需满足训练数据中未包含的新约束(如避障)这一痛点,研究提出CS-Pi(Constraint-Satisfying Flow Policy Sampling)方法。该方法旨在解决重训练不现实、梯度引导易陷入局部均值、纯拒绝采样效率低及纯投影破坏策略分布的问题。实验显示,CS-Pi在7-DoF货架抓取任务中实现81%的安全成功率,在Franka机器人零样本硬件部署中达到55%,相比传统投影法分别提升26和15个百分点,同时确保了动作的策略合理性。
CS-Pi核心采用“绘制-投影-重加噪”(draw-project-renoise)采样机制:先将采样动作投影至最近安全点,再根据投影距离按比例回填噪声并重新运行原始策略,直至生成既安全又符合策略分布的动作。该架构无需修改原策略权重,仅替换推理期投影步骤即可扩展至连续成本函数。技术验证覆盖2D诊断、7-DoF机械臂货架操作、方向偏好约束及Franka真机零样本部署四个场景,有效克服了梯度引导在双模态分布下的平均化失效问题。
关键看点
- 采样机制:采用draw-project-renoise流程,投影后按位移比例回填噪声并重跑策略,兼顾安全性与分布合理性
- 性能指标:7-DoF货架任务安全成功率81%,Franka零样本部署达55%,较纯投影法分别提升26%和15%
- 零样本适配:无需重训练即可在推理期强制执行新约束,支持从离散避障扩展至连续成本函数
- 痛点解决:克服梯度引导在多模态分布下取均值导致穿障的问题,避免纯拒绝采样在安全域稀疏时的等待
工程与产业观察
该工作为具身智能模型部署提供了关键的推理期安全对齐范式。在不微调大模型前提下,通过数学采样修正解决Sim-to-Real长尾安全问题,对工业机械臂在非结构化环境中的零样本落地具有直接工程参考价值,降低了安全合规的边际成本。
英文摘要 (Abstract)
A flow-matching policy deployed in the field must often satisfy a constraint that was absent from its demonstrations, such as avoiding an obstacle it has never seen. Retraining for every new constraint is impractical, so the constraint must be enforced at inference time. Enforcing it must not discard what the policy learned: an action the policy would rarely sample may avoid the obstacle yet fail the task. We therefore want actions that are both safe, satisfying the constraint, and plausible, remaining likely under the policy. Gradient guidance, the most common approach, steers noisy states, so it must evaluate its cost on the policy's running estimate of the clean action; when the policy is uncertain between passing left or right of an obstacle, this estimate is their average and can pass straight through it. We instead build on rejection and projection. Rejection redraws actions until one is safe: every output is plausible, but the wait is intractable when safe actions are rare. Projection moves an action to the nearest safe one: it is efficient, but the result need not be plausible. Our key insight is that flow-matching sampling lets us combine the two, keeping projection's efficiency and rejection's plausibility. CS-Pi, our draw-project-renoise sampler, projects each draw to the nearest safe action, adds noise back scaled to how far the projection moved it, and re-runs the unmodified policy until a draw is safe. Replacing only the projection step extends CS-Pi to continuous costs. Across a 2D diagnostic, a 7-DoF shelf pick-and-place, an orientation preference, and a zero-shot Franka deployment, CS-Pi reaches 81% safe success on the shelf and 55% on hardware, against 55% and 40% for projection, while staying plausible.
作者:Soumojit Bhattacharya, Oliver Kroemer, Shubham Tulsiani
外部来源
本页事实对应的公开页面,以原文为准。(点击将离开本站)