论文/arXiv 2610.09763·21 天前

超越策略支持：面向自动驾驶的交互约束离线强化学习

针对自动驾驶离线强化学习中的交互分布偏移，提出ICDP，并在nuPlan、Interplan和真实卡车实验中验证。

AI 整理

• 本体 SDK 与控制
• 数据与采集
• 具身模型

原文链接：Beyond Policy Support: Interaction Constrained Offline Reinforcement Learning for Autonomous Driving（arXiv 2610.09763 · 发布日期：2026-10-07）

Beyond Policy Support: Interaction Constrained Offline Reinforcement Learning for Autonomous Driving

针对自动驾驶离线强化学习中的交互分布偏移，提出ICDP，并在nuPlan、Interplan和真实卡车实验中验证。

核心要点

• 离线强化学习可用固定数据改进策略，但分布偏移会使弱支持动作的价值估计不可靠。
• 论文指出交互环境中仅约束自车动作不足，提出交互分布偏移IDS问题。
• ICDP将联合支持退化分解为自车支持与交互支持，并用对比密度比估计恢复后者。
• 闭环评估显示ICDP抑制高价值但交互不受支持的轨迹选择，并提升交互关键场景表现。

验证范围与局限：验证范围包括nuPlan、Interplan闭环评估和真实卡车实验，面向交互关键驾驶场景。

英文摘要 (Abstract)

Offline reinforcement learning enables reward-driven policy improvement from fixed datasets without requiring online exploration, making it particularly attractive in safety-critical domains. A central challenge, however, is distribution shift: policy optimization may favor actions that are weakly supported by the offline data, rendering value estimates unreliable. Existing approaches primarily control this shift in the policy's own action space. In interactive environments such as autonomous driving, this can be insufficient: a candidate ego trajectory may remain well supported under the marginal behavior distribution while being poorly supported jointly with the surrounding-agent behavior observed in the logged interaction. We refer to this degradation in interaction support as \emph{interaction distribution shift} (IDS), and introduce \emph{Interaction-Constrained Drive Policy} (ICDP), an offline reinforcement learning framework that explicitly controls interaction-level distribution shift. Starting from the joint data distribution over ego and surrounding-agent futures, we show that joint-support degradation decomposes exactly into an ego-support component and a residual interaction-support component. We recover the latter through contrastive density-ratio estimation, isolating interaction compatibility without explicit joint-density modeling, surrounding-agent prediction, or rollouts in reactive simulators or learned world models during policy optimization. Closed-loop evaluations on nuPlan, Interplan and real-world truck experiments show that ICDP suppresses high-value yet interaction-unsupported trajectory selections and improves performance in interaction-critical driving scenarios. Project webpage: https://mahmoud-selim.github.io/ICDP/

作者：Mahmoud Selim, Cristina Cipriani, Karl Henrik Johansson

来源

本页事实对应的公开页面，以原文为准。

1. arXiv 2610.09763：Beyond Policy Support: Interaction Constrained Offline Reinforcement Learning for Autonomous Driving访问 2026-10-21