# 通过 Fixed-point Flow Matching 的强化学习

> 原文链接：https://robotics.hellobike.com/papers/2026-10-09-2610-12848
> 频道：papers | 更新日期：2026-10-09 | 标签：arxiv, cs.RO
> 核心摘要：该文提出无数据的 FFM，用固定点迭代训练流策略，并在离线到在线和在线RL中验证。

# 通过 Fixed-point Flow Matching 的强化学习

::: v-pre
> 原文链接：[Reinforcement Learning via Fixed-point Flow Matching](https://arxiv.org/abs/2610.12848)（arXiv 2610.12848 · 发布日期：2026-10-09）

**Reinforcement Learning via Fixed-point Flow Matching**

该文提出无数据的 FFM，用固定点迭代训练流策略，并在离线到在线和在线RL中验证。

## 核心要点

- 最大熵和行为约束RL中的最优策略为Boltzmann分布，复杂多峰分布采样较困难。
- FFM利用目标流恒等式，将actor更新表述为固定点迭代，用critic动作梯度作回归目标。
- 作者证明最优策略是该迭代的固定点，并通过大量实验展示其有效性。
- FFM可微调预训练流策略，也可在在线RL中从头高效、准确训练流策略。

**验证范围与局限**：验证范围为离线到在线设置和在线RL从头训练；摘要未说明具体任务或平台。

## 英文摘要 (Abstract)

In maximum entropy and behavior-constrained reinforcement learning (RL), the optimal policy is given by a Boltzmann distribution. Generating samples from this distribution can be challenging, as it is often complex and multimodal. While flow policies trained with flow matching provide a powerful framework for representing such distributions, they traditionally rely on access to training data, which is generally unavailable in online RL. To address this limitation, we propose fixed-point flow matching (FFM), a data-free procedure for training and fine-tuning flow policies in RL. FFM leverages the target flow identity to formulate the actor update as a fixed-point iteration, utilizing a flow-matching-style regression objective where the critic's action gradients serve as regression targets. We show that the optimal policy is a fixed point of this iteration. Through extensive experiments, we demonstrate that FFM effectively fine-tunes pretrained flow policies in the offline-to-online setting and enables efficient and accurate training of flow policies from scratch in online RL. These results establish fixed-point flow matching as a principled approach for learning expressive flow-based policies without requiring training data.

作者：Denis Blessing, Onur Celik, Gerhard Neumann
:::

## 原始参考来源 (Sources)

- [arXiv 2610.12848：Reinforcement Learning via Fixed-point Flow Matching](https://arxiv.org/abs/2610.12848)