外观
论文arXiv 2609.142611 个月前
VGFM：通过 Flow Matching 中的稠密价值引导实现富表达力机器人策略
原文链接：VGFM: Expressive Robot Policies via Dense Value Guidance in Flow Matching（arXiv 2609.14261 · 发布日期：2026-09-13）
VGFM: Expressive Robot Policies via Dense Value Guidance in Flow Matching
VGFM在基于Flow Matching的离线RL策略中加入稠密价值引导，并在OGBench运动和操作任务上验证长时程目标控制。
核心要点
- 大规模离线机器人数据推动策略学习，但多步生成式actor的策略改进仍具挑战。
- VGFM将策略参数化为动作空间的条件Flow Matching模型，使中间流步骤产生有效动作。
- 该设计可在随机流时间用标准离线RL critic施加价值引导，避免BPTT和额外算法开销。
- 在OGBench机器人运动与操作任务中，VGFM在严格评估下取得强性能，且仅需少量超参调节。
验证范围与局限：验证范围为OGBench中的机器人运动和操作任务。
英文摘要 (Abstract)
Recent robot learning paradigms increasingly rely on large offline datasets of robotic interactions to train control policies. Expressive generative models enable rich and multimodal action representations, expanding the capability of this paradigm for complex robotic control. However, policy improvement with multi-step generative actors remains challenging. In offline reinforcement learning (RL), incorporating value-based objectives along generative trajectories often introduces substantial training complexity, including backpropagation through time (BPTT), auxiliary architectures, or distillation losses. We propose Value-Guided Flow Matching (VGFM), a scalable offline RL framework that enables dense value-guided shaping within a flow-based policy while avoiding BPTT and additional algorithmic overhead. VGFM parameterizes the policy as a conditional flow-matching model in action (x-prediction) space, ensuring that each intermediate flow step produces a valid robot action that can be directly evaluated by a standard offline RL critic. This design allows value guidance to be applied at randomly sampled flow times without differentiating through the entire generative trajectory, while preserving inference-time flexibility by varying the discretization of the underlying flow ODE without retraining. Evaluated on robotic locomotion and manipulation tasks in OGBench, VGFM achieves strong performance across a wide range of tasks under rigorous evaluation protocols. With minimal hyperparameter tuning, these results demonstrate that VGFM provides a simple, scalable, and effective approach for expressive policy learning in long-horizon, goal-oriented robotic control.
作者：Prajwal Koirala, Mark Campbell
来源
本页事实对应的公开页面，以原文为准。