外观
论文HF Daily Papers · arXiv 2610.343563 天前
RynnVLA-Latent：4B参数VLA模型利用3D隐动作预训练实现多平台高成功率操作
本文由 AI 根据公开内容整理，可能有误，请以原文为准：RynnVLA-Latent: Scaling Robotic Manipulation Foundation Models with 3D-Grounded Latent Actions（HF Daily Papers · arXiv 2610.34356）
核心内容
针对机器人VLA模型受限于动作标注稀缺及控制空间不兼容问题，研究团队发布RynnVLA-Latent框架。该方案核心RynnLAM模块能从视觉转换中学习连续3D接地隐动作，无需原生机器人控制信号。基于30.1K小时人类、机器人及仿真混合视频数据，系统生成7.4552亿个隐动作描述符作为预训练监督源，有效突破了原生动作标签的数据瓶颈。
技术层面采用残差未来特征预测、相机补偿3D光流监督及相机运动解耦机制，确保表征捕获控制相关运动而非仅外观变化。模型通过Flow Matching预测隐动作块，再适配至具体本体。4B参数策略在LIBERO和LIBERO-Plus上分别达98.8%和84.7%成功率，VLABench五赛道平均51.9%，RoboTwin 2.0双臂基准88.6%，且在六项真实世界灵巧手与移动抓取任务中刷新最高成功率记录。
关键看点
- 数据规模：处理30.1K小时异构交互视频，生成7.4552亿个3D接地隐动作描述符用于策略预训练
- 基准性能：4B参数模型在LIBERO达98.8%成功率，RoboTwin 2.0双臂任务平均成功率88.6%
- 核心技术：引入残差未来特征预测与相机补偿3D光流监督，解耦相机运动以提取控制相关表征
- 实机验证：在灵巧手与移动夹爪等六项真实操作任务中取得对比方法最高成功率，验证跨本体泛化力
工程与产业观察
该工作验证了利用海量无标注视频构建3D隐动作空间的有效性，解决了具身智能预训练数据匮乏痛点。通过解耦视觉外观与控制语义，实现了从视频到多本体策略的零样本迁移，为大规模基础模型在异构机器人平台的落地提供了可扩展的数据工程范式。
英文摘要 (Abstract)
Scaling vision-language-action (VLA) models is constrained by scarce action-labeled demonstrations and incompatible control spaces across robot embodiments. We present RynnVLA-Latent, a framework that turns heterogeneous interaction videos into scalable supervision for robot policy pretraining. At its core, RynnLAM learns continuous, 3D-grounded latent actions from visual transitions without requiring native robot controls. By combining residual future-feature prediction, camera-compensated 3D-flow supervision, and camera-motion decoupling, RynnLAM encourages representations that capture control-relevant motion rather than appearance changes alone. We apply RynnLAM to 30.1K hours of human, robot, and simulation videos, producing 745.52M latent-action descriptors. RynnVLA-Latent then learns to predict latent-action chunks from visual observations and language through flow matching, before adapting its pretrained backbone and action expert to embodiment-specific robot actions. The resulting 4B policy achieves 98.8% success on LIBERO, 84.7% on LIBERO-Plus without benchmark-specific adaptation, 51.9% average success across five VLABench tracks, and 88.6% average success on the RoboTwin 2.0 bimanual benchmark. It also achieves the highest observed success rates among the compared methods on six real-world manipulation tasks across dexterous-hand and mobile-gripper platforms. Together with representation ablations and data-scaling results, these findings demonstrate the potential of latent-action supervision for scaling robot policy learning beyond native action-labeled data.
Hugging Face Daily Papers 热度：9 upvotes 作者：Haoyu Zhao, Xingyue Zhao, Zhian Su, Bohan Hou, Yinzhou Tang, Fengyi Wang, Kehan Li, Yuming Jiang, Siteng Huang, Xin Li, Zhenyu Zeng, Mingxiu Chen, Deli Zhao, Zhongyu Li
来源
本页事实对应的公开页面，以原文为准。