论文arXiv 2611.07229今天

Spatial-UMI-Dream：5B世界动作模型解耦几何与操作

提出5B参数Spatial-UMI-Dream模型，基于605小时RGB-D数据预训练，显式建模度量几何以解决UMI视角纠缠问题。在仿真与真机任务中成功率分别达79.1%和85%，显著优于基线并验证跨具身泛化能力。

AI 整理

仿真与训练
数据与采集
具身模型

本文由 AI 根据公开内容整理，可能有误，请以原文为准：Spatial-UMI-Dream: Scaling World Action Models with Metric Geometry for UMI Manipulation（arXiv 2611.07229）

核心内容

针对通用操作接口（UMI）因移动近距离相机导致物体运动与相机位姿耦合、缺乏稳定外部视角及显式度量几何的痛点，研究团队发布Spatial-UMI-Dream。该5B参数空间世界动作模型（WAM）基于605小时RGB-D UMI数据，采用“纯世界适应+联合世界-动作学习”两阶段预训练策略，旨在统一预测未来RGB、鱼眼对齐度量深度及动作序列，为机器人无关的操作演示采集提供精确的空间几何支撑。

技术层面，模型利用立体视觉衍生深度信息，在统一的双夹爪RGB-D表征中显式编码场景尺度与空间关系。配套发布的RoboTwin-UMI仿真基准包含20项任务，支持对环境与具身变化的解耦评估。实验显示，该模型在域内成功率79.10%，跨具身与环境迁移下达72.15%，较最强基线分别提升4.65和10.30个百分点；6项真机任务平均成功率85.0%，超基线24.2%。消融证实RGB-D预训练比纯RGB在域内成功率高5.75%。

关键看点

模型架构：5B参数WAM经605h RGB-D数据两阶段预训练，联合预测未来RGB、鱼眼对齐度量深度与动作，解决UMI视角纠缠难题
评测基准：推出RoboTwin-UMI仿真平台含20项任务，支持腕部鱼眼观测，实现环境与具身变化下策略泛化能力的解耦零样本评估
性能指标：仿真域内成功率79.10%，跨域迁移72.15%；真机6任务平均成功率85.0%，较最强基线提升24.2%，具备最小具身迁移衰减
数据模态：消融实验表明RGB-D预训练相比纯RGB使域内成功率提升5.75%，证实显式度量几何对精准操作策略学习的关键作用

工程与产业观察

该工作证明了在UMI等动态第一人称视角下，通过大规模RGB-D预训练显式恢复度量几何是提升操作策略泛化性的关键路径。其提出的世界动作模型范式与解耦评测基准，为解决具身智能中“演示数据-物理执行”的Sim-to-Real鸿沟提供了可复现的工程参照，尤其对依赖低成本手持采集数据的团队具有直接指导意义。

英文摘要 (Abstract)

The Universal Manipulation Interface (UMI) enables scalable demonstration collection independently of a specific robot. However, its moving, close-range gripper cameras entangle camera and object motion without a stable external view, leaving the metric geometry needed for precise manipulation implicit in RGB. We introduce Spatial-UMI-Dream, a 5B spatial World Action Model (WAM) pretrained on 605 hours of RGB-D UMI data through two-stage training -- world-only adaptation followed by joint world-action learning -- to jointly predict future RGB, fisheye-aligned metric depth, and actions. Stereo-derived depth makes scene scale and spatial relationships explicit within a unified dual-gripper RGB-D representation. Furthermore, evaluating UMI policies requires testing whether UMI's robot-independent demonstration collection translates into policy generalization across robot embodiments, alongside robustness to environmental changes. We introduce RoboTwin-UMI, a twenty-task simulation benchmark built on RoboTwin with UMI wrist-mounted fisheye observations, enabling controlled, repeatable evaluation of UMI policies under environment and embodiment changes separately and jointly without additional fine-tuning. On RoboTwin-UMI, Spatial-UMI-Dream achieves 79.10% in-domain success and 72.15% under combined embodiment and environment shifts, exceeding the strongest evaluated baseline in each setting by 4.65 and 10.30 percentage points, respectively. It also exhibits the smallest success-rate drop under embodiment transfer among the evaluated policies. In a matched pretraining ablation, RGB-only pretraining leads to a 5.75-percentage-point decrease in in-domain success compared with RGB-D pretraining. Across six real-world tasks, Spatial-UMI-Dream achieves 85.0% average success, exceeding the strongest baseline by 24.2 percentage points. Project page: https://weiduoyuan.com/spatial-umi-dream/

作者：Weiduo Yuan, Ze Yuan, Sihan Liu, Jionghao Wang, Liang Heng, Zhenyu Zhao, Yue Wang