外观
论文arXiv 2610.1766715 天前
PrT-Flow：用于强化学习的概率可处理 Flow Matching
原文链接：PrT-Flow: Probability-Tractable Flow Matching for Reinforcement Learning（arXiv 2610.17667 · 发布日期：2026-10-13）
PrT-Flow: Probability-Tractable Flow Matching for Reinforcement Learning
PrT-Flow用可逆JTT与CET实现可求动作概率的多步生成，并在Robomimic Lift和OGBench Cube-Triple上验证。
核心要点
- 扩散模型和Flow Matching能表示灵活动作分布，但许多强化学习算法还需要动作概率。
- JTT使每个采样步可逆并可高效计算Jacobian行列式，CET监督采样时的单步位移。
- 在Robomimic Lift模仿学习达96%成功率，在OGBench Cube-Triple在线训练后达96%。
- 结果表明可求密度的可逆网络可学习机器人策略并支持SAC式actor-critic训练，代码已开源。
验证范围与局限：验证范围为Robomimic Lift模仿学习和OGBench Cube-Triple在线强化学习任务。
英文摘要 (Abstract)
Diffusion models and Flow Matching can represent flexible distributions over robot actions, but many reinforcement learning algorithms also require the probability of each action. We introduce PrT-Flow (Probability-Tractable Flow Matching), which computes probability density while retaining multimodal, multi-step action generation. Its Jacobian-Tractable Transformer (JTT) makes each sampling step invertible and enables efficient computation of its Jacobian determinant. We train these finite steps with Conditional Euler-step Training (CET), which supervises their displacement over one step during sampling rather than their instantaneous velocity. We prove that, under certain assumptions, CET has the same minimizers as fitting the Euler-solver updates of Flow Matching. PrT-Flow achieves 96% success on Robomimic Lift after 300 epochs of imitation learning, matching the diffusion policy reference, and 96% success on OGBench Cube-Triple after 600k online steps. These results demonstrate that invertible networks with tractable probability densities can learn effective robot policies and support SAC-style actor-critic training. Our code is available at https://github.com/Sunlight-zero/PrT-Flow .
作者：Congzhong Sun, Ning Tan
来源
本页事实对应的公开页面，以原文为准。