外观
论文HF Daily Papers · arXiv 2610.314746 天前
SkeLAM：骨架引导潜在动作模型提升机器人世界模型可控性
本文由 AI 根据公开内容整理，可能有误，请以原文为准：Skeleton-Grounded Latent Action Model for Robot World Models（HF Daily Papers · arXiv 2610.31474）
核心内容
针对现有潜在动作模型将机器人运动与环境响应混淆导致控制对应困难的问题，研究提出SkeLAM（Skeleton-Grounded Latent Action Model）。该方法将潜在动作解耦为由机器人骨架监督的运动学分量与保留视觉动态的残差分量，并通过跨模态交换重建机制确保RGB与骨架表征在视频生成中可互换。实验显示，在DROID数据集上相比DreamDojo，SkeLAM使末端执行器平均位移误差（ADE）降低19.5%，感知相似度指标LPIPS优化11.1%。
系统架构采用独立条件通路融合两类分量，结合分支Dropout降低对未来残差信息的依赖；轻量级适配器将真实机器人动作映射至运动学空间，微调后支持基于当前观测与动作的未来帧生成。Motion Probing验证了运动学分量的专一性，定性结果证实生成的Rollouts可实现零样本人到机器人动作迁移。策略评估表明，模型预测的成功率与真实机器人执行实测值高度一致，验证了其在Sim-to-Real闭环中的可靠性。
关键看点
- 动作解耦机制：将潜在动作分解为骨架监督的运动学分量与视觉残差分量，解决重构驱动方法中机器人与环境动态耦合难题
- 量化性能提升：在DROID基准测试中，末端执行器ADE较DreamDojo降低19.5%，LPIPS指标优化11.1%，生成精度显著增强
- 零样本迁移能力：定性实验证实生成视频支持Zero-shot Human-to-Robot动作迁移，无需额外标注即可复用人类演示数据
- 虚实评估对齐：策略评估显示模型预测成功率与真机执行实测值高度吻合，验证世界模型作为离线评估器的有效性
工程与产业观察
SkeLAM通过引入显式骨架先验解决了视频生成模型中“动作-状态”语义模糊的工程痛点，为具身智能提供了更精准的Sim-to-Real桥梁。其解耦设计不仅提升了生成质量，更关键的是实现了人类视频数据向机器人策略的零样本迁移，大幅降低了高质量具身数据的采集成本，对构建通用机器人基础模型具有重要参考价值。
英文摘要 (Abstract)
Robot world models simulate how environments evolve in response to robot actions. Latent action models enable learning dynamics from videos by encoding visual transitions as proxy actions. However, reconstruction-driven latent actions mix robot motion with object responses and other scene changes, complicating their correspondence with robot controls. We propose SkeLAM, a skeleton-guided latent action model for controllable robot world modeling. SkeLAM decomposes latent actions into a kinematic component grounded by robot skeleton supervision and a residual component retaining complementary visual dynamics. Cross-modal swap reconstruction encourages RGB-derived and skeleton-derived kinematic representations to be interchangeable during video reconstruction. The world model incorporates both components through separate conditioning pathways, while branch dropout reduces its dependence on future-derived residual information. A lightweight adapter maps robot actions to the kinematic space, and subsequent world-model fine-tuning adapts generation to these predicted conditions. At inference, the model generates future observations from the current observation and robot actions. On DROID, SkeLAM reduces end-effector ADE by 19.5% and LPIPS by 11.1% compared with DreamDojo. Motion probing supports kinematic specialization, while qualitative results demonstrate zero-shot human-to-robot motion transfer in generated rollouts. Policy evaluation shows agreement between predicted success rates and those measured in real robot executions.
Hugging Face Daily Papers 热度：7 upvotes 作者：Qi Lv, Jiayi Lin, Yuxin Jiang, Jianzong Wang, Xiang Deng, Mike Shou Zheng
来源
本页事实对应的公开页面，以原文为准。