# RFPO：用于具身控制的 Rectified Flow 策略优化

> 原文链接：https://robotics.hellobike.com/papers/2026-10-07-2610-10453
> 频道：papers | 更新日期：2026-10-07 | 标签：arxiv, cs.RO
> 核心摘要：RFPO针对flow策略少步积分退化，用在线Reflow与PPO监督优化，在多种机器人及真机运动中验证。

# RFPO：用于具身控制的 Rectified Flow 策略优化

::: v-pre
> 原文链接：[RFPO: Rectified Flow Policy Optimization for Embodied Control](https://arxiv.org/abs/2610.10453)（arXiv 2610.10453 · 发布日期：2026-10-07）

**RFPO: Rectified Flow Policy Optimization for Embodied Control**

RFPO针对flow策略少步积分退化，用在线Reflow与PPO监督优化，在多种机器人及真机运动中验证。

## 核心要点

- 基于flow的连续控制表达力强，但ODE迭代积分开销大，直接减少步数会产生few-step discretization gap。
- RFPO在on-policy学习中用奖励感知在线Reflow修正学生传输路径，并结合冻结Gaussian PPO控制器监督。
- 部署策略是单个flow student并用一步Euler执行，在四种机器人上保持接近64步的控制表现。
- 在Unitree Go2上一步执行保留98.5%的64步奖励，平均推理延迟由4.39 ms降至0.08 ms；代码和网站已公开。

**验证范围与局限**：摘要仅说明在Unitree Go2、Boston Dynamics Spot、Unitree H1、Unitree G1上评估，并通过真机实验验证稳定一步运动。

## 英文摘要 (Abstract)

Flow-based policies provide an expressive framework for continuous robot control, but their iterative ODE integration incurs substantial inference cost. Naively reducing the integration budget can severely degrade control, since policies optimized under full-step execution are not explicitly constrained to remain reliable under coarse numerical integration. We refer to this mismatch as the few-step discretization gap. To address this problem, we introduce RFPO, a flow-policy optimization framework for reliable few-step execution. Reward-aware online Reflow rectifies student-induced transport paths during on-policy learning, making the resulting policy more robust to coarse integration. A frozen Gaussian PPO controller supplies complementary action-space supervision at full and intermediate integration budgets, while the deployed policy remains a single flow student executed with one Euler step. Across Unitree Go2, Boston Dynamics Spot, Unitree H1, and Unitree G1, RFPO consistently preserves full-step control performance under one-step execution, with one-step returns remaining within 2.4% of their corresponding 64-step values across both zero and random initialization. On Unitree Go2, one-step execution retains 98.5% of the 64-step reward while reducing onboard mean inference latency from 4.39 ms to 0.08 ms, yielding a 54.9x speedup. Real-robot experiments further validate stable one-step locomotion. Code: https://github.com/AIGeeksGroup/RFPO. Website: https://aigeeksgroup.github.io/RFPO.

作者：Ting Huang, Lisiyu Pan, Haoyu Wang, Zeyu Zhang, Siyuan Qian, Yanjun Li, Yandong Guo, Boxin Shi, Hao Tang
:::

## 原始参考来源 (Sources)

- [arXiv 2610.10453：RFPO: Rectified Flow Policy Optimization for Embodied Control](https://arxiv.org/abs/2610.10453)