外观
论文arXiv 2406.092462024-06-13
OpenVLA开源7B视觉语言动作模型，操控成功率超RT-2-X
本文由 AI 根据公开内容整理，可能有误，请以原文为准：OpenVLA: An Open-Source Vision-Language-Action Model（arXiv 2406.09246）
核心内容
针对现有视觉语言动作（VLA）模型闭源及微调效率低的问题，研究团队推出OpenVLA。该模型拥有7B参数，基于97万条真实世界机器人演示数据训练，旨在通过大规模预训练与高效微调改变机器人技能习得范式。实测显示，OpenVLA在29项跨本体操控任务中的绝对成功率比55B参数的闭源RT-2-X高出16.5%，且参数量仅为其七分之一，验证了小规模开源模型在通用操控领域的潜力。
技术架构上，OpenVLA融合Llama 2语言模型与DINOv2、SigLIP双视觉编码器特征。在多物体复杂场景微调中，其表现比从头训练的Diffusion Policy高20.4%，展现出强语言接地能力。工程落地方面，模型支持通过LoRA在消费级GPU上微调，并经量化后保持下游成功率无损。团队已开源模型权重、PyTorch代码库及微调Notebook，原生支持Open X-Embodiment数据集的大规模训练。
关键看点
- 模型性能：7B参数OpenVLA在29项跨本体任务中绝对成功率超55B闭源RT-2-X达16.5%
- 微调优势：多任务场景下微调泛化能力显著，比从头训练的Diffusion Policy成功率高20.4%
- 架构设计：基于Llama 2结合DINOv2与SigLIP双视觉编码器，使用97万条真实演示数据预训练
- 部署门槛：支持消费级GPU LoRA微调及量化推理，开源权重与代码并兼容Open X-Embodiment
工程与产业观察
OpenVLA证明了7B级开源VLA在通用操控上可超越大参数闭源模型，确立了“高质量真实数据+高效微调”的技术路线。其对消费级算力的适配及完整工具链开源，大幅降低了具身智能算法研发门槛，为端侧部署与社区复现提供了关键基准。
英文摘要 (Abstract)
Large policies pretrained on a combination of Internet-scale vision-language data and diverse robot demonstrations have the potential to change how we teach robots new skills: rather than training new behaviors from scratch, we can fine-tune such vision-language-action (VLA) models to obtain robust, generalizable policies for visuomotor control. Yet, widespread adoption of VLAs for robotics has been challenging as 1) existing VLAs are largely closed and inaccessible to the public, and 2) prior work fails to explore methods for efficiently fine-tuning VLAs for new tasks, a key component for adoption. Addressing these challenges, we introduce OpenVLA, a 7B-parameter open-source VLA trained on a diverse collection of 970k real-world robot demonstrations. OpenVLA builds on a Llama 2 language model combined with a visual encoder that fuses pretrained features from DINOv2 and SigLIP. As a product of the added data diversity and new model components, OpenVLA demonstrates strong results for generalist manipulation, outperforming closed models such as RT-2-X (55B) by 16.5% in absolute task success rate across 29 tasks and multiple robot embodiments, with 7x fewer parameters. We further show that we can effectively fine-tune OpenVLA for new settings, with especially strong generalization results in multi-task environments involving multiple objects and strong language grounding abilities, and outperform expressive from-scratch imitation learning methods such as Diffusion Policy by 20.4%. We also explore compute efficiency; as a separate contribution, we show that OpenVLA can be fine-tuned on consumer GPUs via modern low-rank adaptation methods and served efficiently via quantization without a hit to downstream success rate. Finally, we release model checkpoints, fine-tuning notebooks, and our PyTorch codebase with built-in support for training VLAs at scale on Open X-Embodiment datasets.
作者：Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ…
来源
本页事实对应的公开页面，以原文为准。