论文/arXiv 2608.26571·2 个月前

Arrive and Survive：从一位失败信号规模化安全目标条件策略学习

Safe-CRL针对故障终止下CRL高估目标到达价值，仅用一位失败信号和两项校正，在十二个机器人导航与运动任务上验证安全学习。

AI 整理

• 本体 SDK 与控制
• 感知与导航
• 具身模型

原文链接：Arrive and Survive: Scaling Safe Goal-Conditioned Policy Learning from One-Bit Failure Signals（arXiv 2608.26571 · 发布日期：2026-08-27）

Arrive and Survive: Scaling Safe Goal-Conditioned Policy Learning from One-Bit Failure Signals

Safe-CRL针对故障终止下CRL高估目标到达价值，仅用一位失败信号和两项校正，在十二个机器人导航与运动任务上验证安全学习。

核心要点

• 故障终止下，既有CRL忽略被终止移除的概率质量，导致目标到达价值系统性高估。
• 方法引入质量加权InfoNCE与log生存质量得分，分别校正评论器学习和策略优化。
• 在十二个易失败机器人导航与运动任务中，Safe-CRL提升生存率并优于Scaling-CRL。
• 深度Safe-CRL策略表现出复杂失败规避行为，代码已在GitHub链接公开。

验证范围与局限：验证范围为摘要所述十二个易失败机器人导航和运动任务。

英文摘要 (Abstract)

Contrastive reinforcement learning (CRL) scales effectively in goal-conditioned tasks by casting policy learning into a self-supervised contrastive objective. However, in a failure-terminated Markov decision process, established CRL considers pre-failure future goals only when constructing positive samples, without accounting for the probability mass removed by failure termination. Our theoretical analysis shows that this omission induces a systematic overestimation bias in goal-reaching values. Consequently, near-failure trajectories provide disproportionately strong supervision of success despite retaining little future occupancy. Unsafe actions can thereby be reinforced through catastrophic failure bootstrapping, leading to failed policy learning and unsustainable goal-reaching behaviours. To address this problem, we introduce two minimal yet strong corrections: mass-weighted InfoNCE corrects the overweighting of short surviving futures in critic learning, and a log-survival-mass score restores the missing survival mass in policy optimization. The resulting method, Safe Contrastive Reinforcement Learning (Safe-CRL), requires only the one-bit signal provided by failure termination to scale safe goal-conditioned policy learning. Across twelve failure-prone robot navigation and locomotion tasks, Safe-CRL consistently improves survival and substantially outperforms the Scaling-CRL baseline in goal-reaching performance. Additionally, deep Safe-CRL policies exhibit complex failure-avoidance behaviours. This study completes the CRL theory under failure termination and provides a scalable safe RL framework. The code is available via https://github.com/RomainLITUD/safe-crl.

作者：Guopeng Li, Yiyang Duan, Yiru Jiao, Chengcheng Xu

来源

本页事实对应的公开页面，以原文为准。

1. arXiv 2608.26571：Arrive and Survive: Scaling Safe Goal-Conditioned Policy Learning from One-Bit Failure Signals访问 2026-10-21