外观
论文HF Daily Papers · arXiv 2610.36791昨天
RynnWorld-Latent：基于通用潜在动作的跨具身视频世界模型
本文由 AI 根据公开内容整理，可能有误，请以原文为准：RynnWorld-Latent: A Cross-Embodiment World Model Conditioned on Universal Latent Actions（HF Daily Papers · arXiv 2610.36791）
核心内容
针对异构机器人动作空间不兼容及人类视频缺乏控制标签难题，研究团队发布RynnWorld-Latent跨具身视频世界模型。该模型采用RynnLAM引入的3D锚定潜在动作作为共享条件接口，实现人类交互视频、机器人示教及仿真数据的联合预训练。模型基于预训练视频Transformer构建，在包含23个数据集、约5.535亿帧的大规模语料上完成预训练，并通过后训练适配目标机器人本体。
架构融合全局动作Token与身份初始化局部通路以对齐未来视频潜变量。系统提供两种数据生成范式：数字遥操利用目标指令生成演示，潜在动作合成将源运动迁移至编辑后的初始帧并配对重定向伪动作。实验表明，合成数据中期训练提升了LIBERO及LIBERO-Plus基准的策略成功率；在真实机器人平台上，两种生成路径均比纯真实示教训练获得更高任务成功率，验证了通用潜在动作作为跨具身可控视觉动力学接口的有效性。
关键看点
- 统一动作接口：采用3D锚定潜在动作作为共享条件，解决异构本体动作空间不兼容问题，支持多模态数据联合预训练
- 大规模预训练：覆盖23个数据集共5.535亿帧视频，结合全局动作Token与局部通路架构，实现跨具身视觉动力学学习
- 双路数据生成：支持数字遥操与潜在动作合成两种范式，通过运动迁移与伪动作重定向，低成本扩充高质量训练数据
- 策略性能提升：合成数据中期训练提高LIBERO基准成功率，真机测试中双生成路径均优于纯真实示教训练效果
工程与产业观察
该工作验证了“通用潜在动作”作为跨具身世界模型统一接口的工程可行性。通过解耦原生动作空间，算法工程师可利用海量无标签人类视频与异构机器人数据进行联合预训练，配合合成数据增强策略，有效缓解具身智能落地中的数据稀缺与泛化瓶颈，为构建通用视觉动力学基座提供了可扩展的技术范式。
英文摘要 (Abstract)
Learning controllable world dynamics from heterogeneous interaction videos is challenging because robot embodiments use incompatible action spaces, while human videos lack native control labels. We present RynnWorld-Latent, a cross-embodiment video world model that uses the 3D-grounded latent actions introduced by RynnLAM as a shared conditioning interface. This representation enables joint pretraining on human interaction videos, robot demonstrations, and simulation rollouts without requiring a common native action space. Built on a pretrained video transformer, RynnWorld-Latent combines global action tokens with an identity-initialized local pathway that aligns actions with future video latents. We pretrain on 23 datasets comprising approximately 553.5M frames and adapt the model to target robots through post-training. The framework supports two data-generation routes: digital teleoperation uses target robot commands to generate action-video demonstrations, while latent-action synthesis transfers source motion to an edited initial frame and pairs the resulting rollout with retargeted pseudo robot actions. Experiments show improved video prediction metrics on public benchmarks and two real-robot platforms, alongside qualitative action following when different action sequences share the same initial observation. Synthetic mid-training improves policy success rates on LIBERO and LIBERO-Plus, and both generation routes improve real-world policy success rates over real-demonstration-only training in the evaluated settings. These results support universal latent actions as a scalable interface for learning controllable visual dynamics across heterogeneous embodiments.
Hugging Face Daily Papers 热度：8 upvotes 作者：Haoyu Zhao, Yinzhou Tang, Zixuan Wang, Minghao Zhu, Dongchi Huang, Xingyue Zhao, Qin Zhao, Kehan Li, Siteng Huang, Xin Li, Zhenyu Zeng, Mingxiu Chen, Deli Zhao, Zhongyu Li
来源
本页事实对应的公开页面，以原文为准。