外观
论文arXiv 2610.0043828 天前
通过第一视角全身人体数据预训练迈向通用人形机器人移动操作模型
原文链接：Towards a General Humanoid Loco-Manipulation Model via Egocentric Whole-Body Human Data Pretraining（arXiv 2610.00438 · 发布日期：2026-09-30）
Towards a General Humanoid Loco-Manipulation Model via Egocentric Whole-Body Human Data Pretraining
论文提出HumanVerse-500与λ0三阶段训练，学习人类经验迁移，并在SIMPLE和4个真实移动操作任务验证。
核心要点
- 动机：第一视角人类视频可扩展，但全身运动及手物协同监督覆盖有限。
- 方法：用轻量可穿戴系统采集HumanVerse-500，同步第一视角视频与身手运动。
- 训练：λ0先学交互，再用HumanVerse-500协调身体与手，最后适配任务和机器人形态。
- 结果：在SIMPLE和4个真实任务达到最先进性能，并计划发布代码、模型和数据。
验证范围与局限：验证范围包括SIMPLE和4个真实世界移动操作任务。
英文摘要 (Abstract)
Humanoid whole-body manipulation has advanced rapidly, enabling policies to coordinate locomotion, posture, bimanual interaction, and dexterous hand movements. Meanwhile, egocentric human videos provide diverse examples of everyday interactions across objects and scenes, offering scalable supervision without robot operation. However, existing supervision from these videos provides limited coverage of whole-body movement and coordination with hand-object interaction, while obtaining such supervision through humanoid teleoperation is also costly and difficult to scale. We therefore explore how human experience can support scalable learning of humanoid loco-manipulation. To support this study, we introduce HumanVerse-500, a 500-hour dataset of diverse human loco-manipulation behaviors in open-world environments, collected with a lightweight wearable system that synchronizes egocentric video with body and hand motion. Building on this dataset, we develop $λ_0$, a whole-body humanoid vision-language-action policy, through three-stage training that first learns interaction from diverse egocentric datasets, then coordinates body and hand motion using HumanVerse-500, and finally adapts the policy to downstream tasks and robot embodiments. Across these stages, $λ_0$ learns a shared representation space for human experience transfer, while domain-specific interfaces handle differences between human and robot states and actions. We evaluate $λ_0$ on SIMPLE and 4 real-world loco-manipulation tasks, achieving state-of-the-art performance, and further analyze its scaling behavior, generalization, and training-stage contributions to understand how human data support downstream whole-body humanoid control. We will release our code, models, and data to support further research.
作者：Chongyang Xu, Zhao Wu, Jin Chen, Yiming Jiang, Jinhui Ye, Yuming Jiang, Shifeng Zhang, Ziliang Feng, Mu Xu, Yilun Chen
来源
本页事实对应的公开页面，以原文为准。