外观
论文arXiv 2609.357611 个月前
DexRoam：从自我中心全身人类演示中学习移动双臂灵巧操作
原文链接：DexRoam: Learning Mobile Bimanual Dexterous Manipulation from Egocentric Whole-Body Human Demonstrations（arXiv 2609.35761 · 发布日期：2026-09-28）
DexRoam: Learning Mobile Bimanual Dexterous Manipulation from Egocentric Whole-Body Human Demonstrations
DexRoam通过无追踪采集与三阶段对齐，从第一视角全身人类演示学习移动双臂灵巧操作，并在真实世界VLA实验中验证。
核心要点
- 移动双臂灵巧操作需连续协调移动、全身运动与手指灵巧性，机器人演示存在严重瓶颈。
- 系统仅用消费级VR头显和头戴式立体相机采集全身人类操作演示，无需外部相机或运动追踪器。
- 通过形体、动作语义和时间三阶段对齐，将捕获动作映射到机器人动作空间，并与机器人演示共同训练VLA策略。
- 真实世界实验显示，人类演示在不同VLA骨干和训练范式下稳定提升策略学习，消融表明各对齐阶段均必要。
验证范围与局限：验证限于真实世界实验中的不同VLA骨干，摘要未说明具体任务、机器人数量或仿真结果。
英文摘要 (Abstract)
Mobile bimanual dexterous manipulation requires continuous coordination of locomotion, whole-body motion, and finger-level dexterity within a single trajectory, creating a severe robot demonstration bottleneck. Egocentric human demonstrations offer a scalable alternative, but prior approaches ease the transfer by simplifying human motion, discarding exactly the fine-grained, coupled structure such tasks depend on. We present DexRoam, a complete system for learning mobile bimanual dexterous manipulation from human demonstrations, in which whole-body motion remains continuous and coupled throughout the human-to-robot transfer process. To enable scalable collection of whole-body human manipulation demonstrations, we develop a tracker-free capture system using only a consumer VR headset and a head-mounted stereo camera, without external cameras or motion trackers. We then perform three explicit alignment stages---embodiment, action-semantic, and temporal---to map captured motion into the robot action space, preserving fine-grained whole-body motion and allowing human and robot demonstrations to be jointly learned by standard VLA policies. Real-world experiments with different VLA backbones show that human demonstrations consistently improve policy learning across training paradigms, raising average success from 29% to 56% on GR00T N1.7 and from 32% to 57% on pi0.5, while matching robot-only training with half the robot demonstrations. Ablations confirm that each alignment stage is necessary. These results highlight the potential of human demonstrations for scalable whole-body mobile manipulation with preserved fine-grained motion structure.
作者：Rui Zhou, Yibo Yuan, Junkai Zhao, Fangyuan Zhao, Xiaoguang Zhao, Shanghang Zhang, Sirui Han
来源
本页事实对应的公开页面，以原文为准。