论文/arXiv 2610.16618·16 天前

DR-APO：从连续控制到具身导航的解耦表征优势引导策略优化

DR-APO为高斯、扩散和流策略提供解耦表征的优势引导优化，并在MuJoCo与具身导航中验证。

AI 整理

• 本体 SDK 与控制
• 感知与导航
• 仿真与训练

原文链接：DR-APO: Decoupled-Representation Advantage-Guided Policy Optimization from Continuous Control to Embodied Navigation（arXiv 2610.16618 · 发布日期：2026-10-12）

DR-APO: Decoupled-Representation Advantage-Guided Policy Optimization from Continuous Control to Embodied Navigation

DR-APO为高斯、扩散和流策略提供解耦表征的优势引导优化，并在MuJoCo与具身导航中验证。

核心要点

• 现有强化学习方法依赖策略表征，难以统一优化新兴生成式策略。
• 方法通过优势引导目标改进与兼容表征的直接动作拟合，解耦策略改进与表征。
• 五个MuJoCo-v5任务中，高斯策略竞争于SAC，扩散和流策略纯策略聚合表现最佳。
• NavDP后训练将SR在仿真中由70.33%升至89.55%，真机由25%升至65%。

验证范围与局限：验证涵盖五个MuJoCo-v5任务，以及NavDP在仿真和真实导航中的后训练。

英文摘要 (Abstract)

Policy representations in reinforcement learning have expanded beyond Gaussian policies to diffusion and flow-matching based generative policies. However, existing RL methods remain representation-specific, lacking a unified optimization interface for emerging generative policies. In particular, extending likelihood-based objectives to multi-step generative policies couples optimization to their internal generation process, where accumulated likelihood ratios can weaken the update signal. Meanwhile, generative-policy-specific methods often rely on multi-candidate generation and critic-based selection, improving overall return while providing limited improvement to the policy itself. We introduce DR-APO (Decoupled-Representation Advantage-Guided Policy Optimization), which decouples policy improvement from policy representation through advantage-guided target improvement and representation-compatible direct action fitting, providing a shared optimization principle for Gaussian, diffusion, and flow policies. Across five MuJoCo-v5 tasks, DR-APO remains competitive with SAC for Gaussian policies and achieves the best aggregate policy-only performance for both diffusion and flow-matching policies. NavDP post-training improves the Success Rate (SR) from 70.33% to 89.55% in simulation and from 25% to 65% in real-world navigation. These results demonstrate a unified policy-improvement principle across Gaussian, diffusion, and flow policies, from continuous-control learning to embodied-policy post-training.

作者：Runhua Zhang, Yingjie Niu, Chao Li

来源

本页事实对应的公开页面，以原文为准。

1. arXiv 2610.16618：DR-APO: Decoupled-Representation Advantage-Guided Policy Optimization from Continuous Control to Embodied Navigation访问 2026-10-21