# DR-APO：从连续控制到具身导航的解耦表征优势引导策略优化

> 原文链接：https://robotics.hellobike.com/papers/2026-10-12-2610-16618
> 频道：papers | 更新日期：2026-10-12 | 标签：arxiv, cs.RO
> 核心摘要：DR-APO为高斯、扩散和流策略提供解耦表征的优势引导优化，并在MuJoCo与具身导航中验证。

# DR-APO：从连续控制到具身导航的解耦表征优势引导策略优化

::: v-pre
> 原文链接：[DR-APO: Decoupled-Representation Advantage-Guided Policy Optimization from Continuous Control to Embodied Navigation](https://arxiv.org/abs/2610.16618)（arXiv 2610.16618 · 发布日期：2026-10-12）

**DR-APO: Decoupled-Representation Advantage-Guided Policy Optimization from Continuous Control to Embodied Navigation**

DR-APO为高斯、扩散和流策略提供解耦表征的优势引导优化，并在MuJoCo与具身导航中验证。

## 核心要点

- 现有强化学习方法依赖策略表征，难以统一优化新兴生成式策略。
- 方法通过优势引导目标改进与兼容表征的直接动作拟合，解耦策略改进与表征。
- 五个MuJoCo-v5任务中，高斯策略竞争于SAC，扩散和流策略纯策略聚合表现最佳。
- NavDP后训练将SR在仿真中由70.33%升至89.55%，真机由25%升至65%。

**验证范围与局限**：验证涵盖五个MuJoCo-v5任务，以及NavDP在仿真和真实导航中的后训练。

## 英文摘要 (Abstract)

Policy representations in reinforcement learning have expanded beyond Gaussian policies to diffusion and flow-matching based generative policies. However, existing RL methods remain representation-specific, lacking a unified optimization interface for emerging generative policies. In particular, extending likelihood-based objectives to multi-step generative policies couples optimization to their internal generation process, where accumulated likelihood ratios can weaken the update signal. Meanwhile, generative-policy-specific methods often rely on multi-candidate generation and critic-based selection, improving overall return while providing limited improvement to the policy itself. We introduce DR-APO (Decoupled-Representation Advantage-Guided Policy Optimization), which decouples policy improvement from policy representation through advantage-guided target improvement and representation-compatible direct action fitting, providing a shared optimization principle for Gaussian, diffusion, and flow policies. Across five MuJoCo-v5 tasks, DR-APO remains competitive with SAC for Gaussian policies and achieves the best aggregate policy-only performance for both diffusion and flow-matching policies. NavDP post-training improves the Success Rate (SR) from 70.33% to 89.55% in simulation and from 25% to 65% in real-world navigation. These results demonstrate a unified policy-improvement principle across Gaussian, diffusion, and flow policies, from continuous-control learning to embodied-policy post-training.

作者：Runhua Zhang, Yingjie Niu, Chao Li
:::

## 原始参考来源 (Sources)

- [arXiv 2610.16618：DR-APO: Decoupled-Representation Advantage-Guided Policy Optimization from Continuous Control to Embodied Navigation](https://arxiv.org/abs/2610.16618)