论文/arXiv 2607.25541·3 个月前

P3：用于稳定的基于 VAE 的机器人学习的概率策略传播

针对 VAE 随机潜变量与 PPO 的失配，提出 P3 分布感知优化，并在人形跑酷任务上验证其效率与稳定性。

AI 整理

• 本体 SDK 与控制
• 具身模型

原文链接：P3: Probabilistic Policy Propagation for Stable VAE-Based Robot Learning（arXiv 2607.25541 · 发布日期：2026-07-28）

P3: Probabilistic Policy Propagation for Stable VAE-Based Robot Learning

针对 VAE 随机潜变量与 PPO 的失配，提出 P3 分布感知优化，并在人形跑酷任务上验证其效率与稳定性。

核心要点

• 问题在于单潜变量样本估计概率比和 KL 散度，会使替代损失产生显著方差与偏差。
• P3 将基于矩的概率方法与采样校准结合，用于潜变量不确定性下的稳定高效学习。
• 实验中 P3 将数据效率从 64.6% 提升到超过 96%，收敛步数减少超过 20%。
• 方法在具有挑战性的人形跑酷任务上评估，代码已在 GitHub 开源。

验证范围与局限：验证范围为具有挑战性的人形跑酷任务，摘要未说明是否包含真机实验。

英文摘要 (Abstract)

Variational Autoencoders are widely used to encode high-dimensional and noisy observations in robotics. However, their stochastic latent creates a mismatch with Proximal Policy Optimization (PPO): an effective policy marginalizes over the latent distribution, whereas former implementations estimate its probability ratio and KL divergence using only one latent sample. We identify a fundamental but overlooked theoretical cause: naive single-sample approximations in stochastic latent space induce significant variance and bias in the surrogate loss. To address this, we introduce P^3 (Probabilistic Policy Propagation), a distribution-aware optimization framework for VAE-based policies. $P^3$ couples moment-based probabilistic method for stable and efficient learning with sampling-based calibration for robust policy behavior under latent uncertainty. In our experiments, P^3 boosts data efficiency from 64.6% to >96%, reduces convergence steps by >20%. Furthermore, P^3 is evaluated on challenging humanoid parkour tasks and shows an effective foundation for VAE-based PPO. Code is available at https://github.com/ylyem9x/P3_Open.

作者：Liyun Yan, Jianming Ma, Yang Zhang, S. Fu, Zhanxiang Cao, Keqi Zhu, Yizhi Chen, Yue Gao

来源

本页事实对应的公开页面，以原文为准。

1. arXiv 2607.25541：P3: Probabilistic Policy Propagation for Stable VAE-Based Robot Learning访问 2026-10-21