论文/arXiv 2610.17799·15 天前

超越安全执行：用于自动驾驶的基于价值校准与自适应引导的安全在线强化学习

提出SOAR-Flow，通过CBF-MPC安全过滤、价值校准与自适应引导，对离线初始化驾驶策略进行安全在线微调，并在仿真和UGV上验证。

AI 整理

• 本体 SDK 与控制
• 仿真与训练
• 数据与采集

原文链接：Beyond Safe Execution: Safe Online Reinforcement Learning for Autonomous Driving with Value Calibration and Adaptive Guidance（arXiv 2610.17799 · 发布日期：2026-10-13）

Beyond Safe Execution: Safe Online Reinforcement Learning for Autonomous Driving with Value Calibration and Adaptive Guidance

提出SOAR-Flow，通过CBF-MPC安全过滤、价值校准与自适应引导，对离线初始化驾驶策略进行安全在线微调，并在仿真和UGV上验证。

核心要点

• 在线RL可适应部署变化，但训练中需安全探索，单靠安全过滤难保证稳定改进。
• 方法结合CBF-MPC过滤、Cal-QL式价值校准、成对纠正学习和自适应引导。
• 仿真和UGV结果显示，受保护在线微调提升驾驶表现并降低对安全纠正的依赖。
• 论文还分析了控制步安全执行与辅助学习影响逐渐减弱的条件，并提供演示视频和项目概览。

验证范围与局限：验证范围为仿真和一台未具名UGV上的驾驶在线微调。

英文摘要 (Abstract)

Online reinforcement learning (RL) enables driving policies to adapt to changing deployment conditions, but requires safe exploration throughout training. Safety filtering alone does not ensure stable policy improvement or reduced dependence on corrections. We propose Safe Online Adaptive Refinement (SOAR-Flow), a framework for safe and stable online fine-tuning of offline-initialized Flow-based driving policies. A control barrier function (CBF)--model predictive control (MPC) filter corrects potentially unsafe actions before execution. To support stable policy updates, we introduce Gaussian-return-aware value calibration based on calibrated Q-learning (Cal-QL), using recorded behavior returns and predicted return variability to smooth conservative critic updates and mitigate policy degradation. Paired corrective learning teaches feasible action adjustments, while an intervention-to-go predictor encourages decisions requiring fewer future corrections. Its predictions also regulate guidance strength, allowing recovery supervision to weaken as predicted dependence decreases. Together with gradually reduced calibration regularization, this supports continued task improvement while preserving hard execution constraints. We analyze conditions for safe execution at control steps and diminishing auxiliary learning influences. We evaluate SOAR-Flow in simulation and on an uncrewed ground vehicle (UGV). The results demonstrate improved driving performance and reduced reliance on safety corrections during protected online fine-tuning. The demonstration video and project overview are available at https://lzqw.github.io/SOAR-Flow/.

作者：Zeqiao Li, Yijing Wang, Haoyu Wang, Zheng Li, Zhiqiang Zuo

来源

本页事实对应的公开页面，以原文为准。

1. arXiv 2610.17799：Beyond Safe Execution: Safe Online Reinforcement Learning for Autonomous Driving with Value Calibration and Adaptive Guidance访问 2026-10-21