# CO-RFT：分块离线RL微调VLA模型，30样本提升57%成功率

> 原文链接：https://robotics.hellobike.com/papers/2025-08-04-2508-02219-84b0e321
> 频道：papers | 更新日期：2025-08-04 | 标签：auto, arxiv, cs.RO
> 核心摘要：针对VLA模型微调难题提出CO-RFT算法，结合动作分块与离线强化学习。仅需30至60条演示数据，实测任务成功率提升57%，循环时间缩短22.3%，未见位置泛化率达44.3%。

# CO-RFT：分块离线RL微调VLA模型，30样本提升57%成功率

::: v-pre
> 本文由 AI 根据公开内容整理，可能有误，请以原文为准：[CO-RFT: Efficient Fine-Tuning of Vision-Language-Action Models through Chunked Offline Reinforcement Learning](https://arxiv.org/abs/2508.02219)（arXiv 2508.02219）

## 核心内容

针对视觉-语言-动作（VLA）模型在机器人控制中面临的样本效率低、动作分块兼容差及训练不稳定问题，研究团队提出CO-RFT算法。该方法专为VLA设计，通过全参数模仿学习初始化策略，再利用分块离线强化学习进行优化。在真实机器人环境中，仅使用30至60条少量演示数据，CO-RFT相比传统监督学习方法实现了57%的任务成功率提升和22.3%的循环时间缩减。

技术核心在于扩展时序差分（TD）学习以适配VLA的动作分块特性，构建Chunked RL框架。算法流程上，先通过全参数微调完成骨干网络与策略的IL初始化，随后引入动作分块机制执行离线RL策略优化。实验验证显示，该方法具备显著的位置泛化能力，在训练集未覆盖的新位置仍保持44.3%的成功率，有效解决了小样本条件下VLA模型从仿真到真机部署的鲁棒性瓶颈。

## 关键看点

- **小样本高效微调**：仅需30至60条演示数据即可完成VLA模型优化，大幅降低具身智能数据采集成本与门槛
- **性能显著提升**：真实环境测试中任务成功率较监督学习提升57%，操作循环时间缩短22.3%
- **分块RL架构创新**：扩展时序差分学习以原生支持动作分块，解决VLA模型与强化学习的兼容性难题
- **强位置泛化能力**：在未见过的新位置仍达44.3%成功率，验证了离线RL对空间鲁棒性的增强效果

## 工程与产业观察

CO-RFT证明了在极少样本下通过离线RL微调VLA模型的可行性，其动作分块TD扩展机制为具身大模型落地提供了关键工程范式。对于缺乏海量真机数据的研发团队，该方案显著降低了从预训练模型到实际部署的边际成本，是解决VLA长尾场景适配与操作效率优化的重要参考路径。

## 英文摘要 (Abstract)

Vision-Language-Action (VLA) models demonstrate significant potential for developing generalized policies in real-world robotic control. This progress inspires researchers to explore fine-tuning these models with Reinforcement Learning (RL). However, fine-tuning VLA models with RL still faces challenges related to sample efficiency, compatibility with action chunking, and training stability. To address these challenges, we explore the fine-tuning of VLA models through offline reinforcement learning incorporating action chunking. In this work, we propose Chunked RL, a novel reinforcement learning framework specifically designed for VLA models. Within this framework, we extend temporal difference (TD) learning to incorporate action chunking, a prominent characteristic of VLA models. Building upon this framework, we propose CO-RFT, an algorithm aimed at fine-tuning VLA models using a limited set of demonstrations (30 to 60 samples). Specifically, we first conduct imitation learning (IL) with full parameter fine-tuning to initialize both the backbone and the policy. Subsequently, we implement offline RL with action chunking to optimize the pretrained policy. Our empirical results in real-world environments demonstrate that CO-RFT outperforms previous supervised methods, achieving a 57% improvement in success rate and a 22.3% reduction in cycle time. Moreover, our method exhibits robust positional generalization capabilities, attaining a success rate of 44.3% in previously unseen positions.

作者：Dongchi Huang, Zhirui Fang, Tianle Zhang, Yihang Li, Lin Zhao, Chunhe Xia
:::

## 原始参考来源 (Sources)

- [arXiv 2508.02219：CO-RFT: Efficient Fine-Tuning of Vision-Language-Action Models through Chunked Offline Reinforcement Learning](https://arxiv.org/abs/2508.02219)