# 用于长时程离线目标条件强化学习的 Diffusion Subgoal Planning

> 原文链接：https://robotics.hellobike.com/papers/2026-09-28-2609-34575
> 频道：papers | 更新日期：2026-09-28 | 标签：arxiv, cs.RO
> 核心摘要：针对长时程离线GCRL价值引导不稳定问题，DSP用扩散式子目标生成进行分层规划，并在离线GCRL基准验证。

# 用于长时程离线目标条件强化学习的 Diffusion Subgoal Planning

::: v-pre
> 原文链接：[Diffusion Subgoal Planning for Long-Horizon Offline Goal-Conditioned Reinforcement Learning](https://arxiv.org/abs/2609.34575)（arXiv 2609.34575 · 发布日期：2026-09-28）

**Diffusion Subgoal Planning for Long-Horizon Offline Goal-Conditioned Reinforcement Learning**

针对长时程离线GCRL价值引导不稳定问题，DSP用扩散式子目标生成进行分层规划，并在离线GCRL基准验证。

## 核心要点

- 长时程任务中，稀疏奖励和折扣使目标条件价值函数提供的指导不稳定。
- DSP将高层规划表述为目标条件子目标上的引导生成推断，学习条件与无条件流。
- 通过classifier-free guidance在推理时引入目标偏置，避免高层显式价值引导。
- 在离线GCRL导航和操作基准上优于已有方法，迷宫环境表现尤为突出。

**验证范围与局限**：验证范围为离线GCRL基准中的导航和操作任务，包括需要多步子目标规划的迷宫环境。

## 英文摘要 (Abstract)

Offline goal-conditioned reinforcement learning (GCRL) learns goal-directed policies from reward-free data, but in long-horizon tasks, goal-conditioned value functions often provide unstable guidance due to sparse rewards and discounting. Hierarchical methods partially mitigate this issue via subgoal decomposition; however, high-level decision-making still relies on noise-sensitive value estimates, leading to unstable behavior in complex environments. We address this limitation by proposing \textbf{D}iffusion \textbf{S}ubgoal \textbf{P}lanning (\textbf{DSP}), a diffusion-based framework for high-level subgoal generation. DSP casts high-level planning as guided generative inference over goal-conditioned subgoals and learns both conditional and unconditional flows, enabling classifier-free guidance to introduce a goal-directed bias at inference time. By removing explicit value-based guidance from high-level planning, DSP generates reachable and goal-directed subgoals through a generative model while retaining hierarchical execution. Experiments on offline GCRL benchmarks demonstrate that DSP outperforms prior methods on a range of navigation and manipulation tasks, with particularly strong performance in maze environments that require multi-step subgoal planning.

作者：Hengrui Zhang, Yuhu Cheng, C. L. Philip Chen, Xuesong Wang
:::

## 原始参考来源 (Sources)

- [arXiv 2609.34575：Diffusion Subgoal Planning for Long-Horizon Offline Goal-Conditioned Reinforcement Learning](https://arxiv.org/abs/2609.34575)