# OGPO：用于可恢复视觉-语言-动作模型的离线目标条件策略优化

> 原文链接：https://robotics.hellobike.com/papers/2026-10-09-2610-12914
> 频道：papers | 更新日期：2026-10-09 | 标签：arxiv, cs.RO
> 核心摘要：针对VLA后训练难以从偏离演示轨迹中恢复的问题，OGPO用语义关键状态和目标条件奖励优化策略，并在多个基准上验证。

# OGPO：用于可恢复视觉-语言-动作模型的离线目标条件策略优化

::: v-pre
> 原文链接：[OGPO: Offline Goal-conditioned Policy Optimization for Recoverable Vision-Language-Action Models](https://arxiv.org/abs/2610.12914)（arXiv 2610.12914 · 发布日期：2026-10-09）

**OGPO: Offline Goal-conditioned Policy Optimization for Recoverable Vision-Language-Action Models**

针对VLA后训练难以从偏离演示轨迹中恢复的问题，OGPO用语义关键状态和目标条件奖励优化策略，并在多个基准上验证。

## 核心要点

- 现有轨迹级监督模仿易偏向演示动作路径，对偏离后的任务进展恢复指导有限。
- OGPO自适应重标注语义关键状态，构建目标条件过程奖励，优化到达任务进展状态。
- 在LIBERO和MetaWorld上，OGPO相比标准监督微调持续提升VLA策略性能。
- 在LIBERO-Plus和LIBERO-ReAct评估中，OGPO表现出更强鲁棒性和目标导向恢复能力。

**验证范围与局限**：验证范围为LIBERO、MetaWorld、LIBERO-Plus及提出的LIBERO-ReAct基准。

## 英文摘要 (Abstract)

Vision-language-action (VLA) models have shown strong promise in robotic manipulation, yet their post-training still predominantly relies on trajectory-level supervised imitation over robot demonstrations. While effective for learning feasible actions, this paradigm supervises policies to reproduce expert actions at each step, biasing them toward demonstration-specific action paths and providing limited guidance for recovering toward task-progress states once execution deviates from demonstrated trajectories. To address this limitation, we propose OGPO, an Offline Goal-conditioned Policy Optimization framework for recoverable VLA post-training. Instead of treating offline demonstrations merely as step-wise action labels, OGPO adaptively relabels trajectories with semantic key states and constructs goal-conditioned process rewards to optimize the policy for reaching these task-progress states. In this way, OGPO shifts VLA post-training from trajectory-level action imitation to key-state reachability learning, enabling policies to acquire a more robust ability to complete goal states from feasible off-demonstration states without requiring additional online interaction. Experiments on LIBERO and MetaWorld show that OGPO consistently improves VLA policy performance over standard supervised fine-tuning. Moreover, evaluations on LIBERO-Plus and our proposed LIBERO-ReAct, a mid-execution perturbation benchmark, further demonstrate that OGPO achieves stronger robustness and goal-directed recoverability under distribution shifts and recoverable off-demonstration states.

作者：Xule Gao, Xi Wang, Zehua Zang, Rui Wang, Changwen Zheng, Chuxiong Sun
:::

## 原始参考来源 (Sources)

- [arXiv 2610.12914：OGPO: Offline Goal-conditioned Policy Optimization for Recoverable Vision-Language-Action Models](https://arxiv.org/abs/2610.12914)