# RoboFlamingo框架：轻量微调VLM实现机器人模仿学习SOTA

> 原文链接：https://robotics.hellobike.com/papers/2023-11-02-2311-01378-68523b80
> 频道：papers | 更新日期：2023-11-02 | 标签：auto, arxiv, cs.RO, landmark
> 核心摘要：基于OpenFlamingo构建的RoboFlamingo框架，通过解耦单步视觉语言理解与显式策略头时序建模，仅用语言条件操作数据微调即可超越SOTA。该方案支持开环控制与低算力平台部署，为VLM适配机器人操控提供低成本范式。

# RoboFlamingo框架：轻量微调VLM实现机器人模仿学习SOTA

::: v-pre
> 本文由 AI 根据公开内容整理，可能有误，请以原文为准：[Vision-Language Foundation Models as Effective Robot Imitators](https://arxiv.org/abs/2311.01378)（arXiv 2311.01378）

## 核心内容

针对视觉语言模型（VLM）在机器人操控任务中适配难、算力需求高的问题，研究团队提出RoboFlamingo框架。该方法基于开源OpenFlamingo模型，摒弃了复杂的端到端训练，转而采用轻量化微调策略，仅在语言条件化操作数据集上进行模仿学习。实验表明，RoboFlamingo在测试基准上以显著优势超越现有SOTA方法，验证了利用预训练VLM进行机器人控制的有效性，并揭示了不同预训练模型在操作任务中的行为差异。

技术架构上，RoboFlamingo将系统解耦为两部分：利用预训练VLM处理单步视觉语言理解，同时引入显式策略头（Explicit Policy Head）独立建模序列历史信息。这种分解设计赋予了系统开环控制能力，使其能在低性能计算平台上部署运行。相比传统方法，该框架无需海量机器人专用数据重训VLM主干，大幅降低了微调门槛与算力成本，使开发者能基于通用VLM快速定制机器人操作策略，具备较高的工程落地参考价值。

## 关键看点

- **架构解耦**：分离单步VLM理解与显式策略头时序建模，支持开环控制及低性能边缘端部署
- **高效微调**：仅依赖语言条件操作数据集进行模仿学习微调，无需重训VLM主干即可适配机器人
- **性能突破**：在测试基准上以显著优势超越当前SOTA，验证了通用VLM作为机器人模仿器的有效性
- **开源基座**：基于OpenFlamingo构建，降低了具身智能算法研发门槛，赋能自定义策略开发

## 工程与产业观察

RoboFlamingo证明了“通用VLM+显式策略头”是平衡泛化性与部署效率的有效路径。其解耦设计解决了大模型上机难的痛点，为端侧具身智能提供了低成本微调范式。这对缺乏海量真机数据的团队极具参考价值，表明利用开源VLM资产结合轻量级时序模块，即可快速构建具备竞争力的语言条件操作策略。

## 英文摘要 (Abstract)

Recent progress in vision language foundation models has shown their ability to understand multimodal data and resolve complicated vision language tasks, including robotics manipulation. We seek a straightforward way of making use of existing vision-language models (VLMs) with simple fine-tuning on robotics data. To this end, we derive a simple and novel vision-language manipulation framework, dubbed RoboFlamingo, built upon the open-source VLMs, OpenFlamingo. Unlike prior works, RoboFlamingo utilizes pre-trained VLMs for single-step vision-language comprehension, models sequential history information with an explicit policy head, and is slightly fine-tuned by imitation learning only on language-conditioned manipulation datasets. Such a decomposition provides RoboFlamingo the flexibility for open-loop control and deployment on low-performance platforms. By exceeding the state-of-the-art performance with a large margin on the tested benchmark, we show RoboFlamingo can be an effective and competitive alternative to adapt VLMs to robot control. Our extensive experimental results also reveal several interesting conclusions regarding the behavior of different pre-trained VLMs on manipulation tasks. We believe RoboFlamingo has the potential to be a cost-effective and easy-to-use solution for robotics manipulation, empowering everyone with the ability to fine-tune their own robotics policy.

作者：Xinghang Li, Minghuan Liu, Hanbo Zhang, Cunjun Yu, Jie Xu, Hongtao Wu, Chilam Cheang, Ya Jing, Weinan Zhang, Huaping Liu, Hang Li, Tao Kong
:::

## 原始参考来源 (Sources)

- [arXiv 2311.01378：Vision-Language Foundation Models as Effective Robot Imitators](https://arxiv.org/abs/2311.01378)