# P3：用于稳定的基于 VAE 的机器人学习的概率策略传播

> 原文链接：https://robotics.hellobike.com/papers/2026-07-28-2607-25541
> 频道：papers | 更新日期：2026-07-28 | 标签：arxiv, cs.RO
> 核心摘要：针对 VAE 随机潜变量与 PPO 的失配，提出 P3 分布感知优化，并在人形跑酷任务上验证其效率与稳定性。

# P3：用于稳定的基于 VAE 的机器人学习的概率策略传播

::: v-pre
> 原文链接：[P3: Probabilistic Policy Propagation for Stable VAE-Based Robot Learning](https://arxiv.org/abs/2607.25541)（arXiv 2607.25541 · 发布日期：2026-07-28）

**P3: Probabilistic Policy Propagation for Stable VAE-Based Robot Learning**

针对 VAE 随机潜变量与 PPO 的失配，提出 P3 分布感知优化，并在人形跑酷任务上验证其效率与稳定性。

## 核心要点

- 问题在于单潜变量样本估计概率比和 KL 散度，会使替代损失产生显著方差与偏差。
- P3 将基于矩的概率方法与采样校准结合，用于潜变量不确定性下的稳定高效学习。
- 实验中 P3 将数据效率从 64.6% 提升到超过 96%，收敛步数减少超过 20%。
- 方法在具有挑战性的人形跑酷任务上评估，代码已在 GitHub 开源。

**验证范围与局限**：验证范围为具有挑战性的人形跑酷任务，摘要未说明是否包含真机实验。

## 英文摘要 (Abstract)

Variational Autoencoders are widely used to encode high-dimensional and noisy observations in robotics. However, their stochastic latent creates a mismatch with Proximal Policy Optimization (PPO): an effective policy marginalizes over the latent distribution, whereas former implementations estimate its probability ratio and KL divergence using only one latent sample. We identify a fundamental but overlooked theoretical cause: naive single-sample approximations in stochastic latent space induce significant variance and bias in the surrogate loss. To address this, we introduce P^3 (Probabilistic Policy Propagation), a distribution-aware optimization framework for VAE-based policies. $P^3$ couples moment-based probabilistic method for stable and efficient learning with sampling-based calibration for robust policy behavior under latent uncertainty. In our experiments, P^3 boosts data efficiency from 64.6% to >96%, reduces convergence steps by >20%. Furthermore, P^3 is evaluated on challenging humanoid parkour tasks and shows an effective foundation for VAE-based PPO. Code is available at https://github.com/ylyem9x/P3_Open.

作者：Liyun Yan, Jianming Ma, Yang Zhang, S. Fu, Zhanxiang Cao, Keqi Zhu, Yizhi Chen, Yue Gao
:::

## 原始参考来源 (Sources)

- [arXiv 2607.25541：P3: Probabilistic Policy Propagation for Stable VAE-Based Robot Learning](https://arxiv.org/abs/2607.25541)