# DEFlow：深度平衡流最大熵强化学习

> 原文链接：https://robotics.hellobike.com/papers/2026-10-12-2610-16597
> 频道：papers | 更新日期：2026-10-12 | 标签：arxiv, cs.RO
> 核心摘要：DEFlow用隐式欧拉和平衡系统构建流策略，并在Multi-Goal docking、操作和MuJoCo任务上验证。

# DEFlow：深度平衡流最大熵强化学习

::: v-pre
> 原文链接：[DEFlow: Deep Equilibrium Flow Maximum-Entropy Reinforcement Learning](https://arxiv.org/abs/2610.16597)（arXiv 2610.16597 · 发布日期：2026-10-12）

**DEFlow: Deep Equilibrium Flow Maximum-Entropy Reinforcement Learning**

DEFlow用隐式欧拉和平衡系统构建流策略，并在Multi-Goal docking、操作和MuJoCo任务上验证。

## 核心要点

- 动机是表达性流策略可表示多种高价值动作，但少步采样且保持可处理动作密度仍具挑战。
- 方法以隐式欧拉积分替代显式欧拉采样，将中间动作联合求解为平衡系统。
- 结合隐式微分和离散变量变换公式，使其可在soft actor-critic中离策略训练。
- 实验显示其在多个任务上优于或竞争于基线，但代价是额外的非线性和线性求解。

**验证范围与局限**：验证范围为四个Multi-Goal docking变体、五个操作任务和五个MuJoCo任务。

## 英文摘要 (Abstract)

Expressive flow policies can represent multiple high-value actions, but sampling them with few integration steps while retaining tractable action densities remains challenging. We introduce DEFlow, a deep equilibrium flow policy that replaces explicit Euler sampling with implicit Euler integration and solves all intermediate actions jointly as an equilibrium system. We combine implicit differentiation with a discrete change-of-variables formula for the final-action density, enabling off-policy training within soft actor-critic without backpropagating through solver iterations. DEFlow achieves an average relative improvement of approximately 10% in mean successful coverage over the strongest baseline across four Multi-Goal docking variants. It also achieves the highest mean time-averaged return on four of five manipulation tasks, while remaining broadly competitive across five MuJoCo tasks. These results support implicit flow policies for tasks requiring both diversity and precision, at the cost of additional nonlinear and linear solves. Primary

作者：Ruslan Ageev, Huaze Tang, Runqing Miao, Jiarong Li, Wenbo Ding
:::

## 原始参考来源 (Sources)

- [arXiv 2610.16597：DEFlow: Deep Equilibrium Flow Maximum-Entropy Reinforcement Learning](https://arxiv.org/abs/2610.16597)