论文/arXiv 2610.12848·19 天前

通过 Fixed-point Flow Matching 的强化学习

该文提出无数据的 FFM，用固定点迭代训练流策略，并在离线到在线和在线RL中验证。

AI 整理

• 具身模型

原文链接：Reinforcement Learning via Fixed-point Flow Matching（arXiv 2610.12848 · 发布日期：2026-10-09）

Reinforcement Learning via Fixed-point Flow Matching

该文提出无数据的 FFM，用固定点迭代训练流策略，并在离线到在线和在线RL中验证。

核心要点

• 最大熵和行为约束RL中的最优策略为Boltzmann分布，复杂多峰分布采样较困难。
• FFM利用目标流恒等式，将actor更新表述为固定点迭代，用critic动作梯度作回归目标。
• 作者证明最优策略是该迭代的固定点，并通过大量实验展示其有效性。
• FFM可微调预训练流策略，也可在在线RL中从头高效、准确训练流策略。

验证范围与局限：验证范围为离线到在线设置和在线RL从头训练；摘要未说明具体任务或平台。

英文摘要 (Abstract)

In maximum entropy and behavior-constrained reinforcement learning (RL), the optimal policy is given by a Boltzmann distribution. Generating samples from this distribution can be challenging, as it is often complex and multimodal. While flow policies trained with flow matching provide a powerful framework for representing such distributions, they traditionally rely on access to training data, which is generally unavailable in online RL. To address this limitation, we propose fixed-point flow matching (FFM), a data-free procedure for training and fine-tuning flow policies in RL. FFM leverages the target flow identity to formulate the actor update as a fixed-point iteration, utilizing a flow-matching-style regression objective where the critic's action gradients serve as regression targets. We show that the optimal policy is a fixed point of this iteration. Through extensive experiments, we demonstrate that FFM effectively fine-tunes pretrained flow policies in the offline-to-online setting and enables efficient and accurate training of flow policies from scratch in online RL. These results establish fixed-point flow matching as a principled approach for learning expressive flow-based policies without requiring training data.

作者：Denis Blessing, Onur Celik, Gerhard Neumann

来源

本页事实对应的公开页面，以原文为准。

1. arXiv 2610.12848：Reinforcement Learning via Fixed-point Flow Matching访问 2026-10-21