Skip to content

论文arXiv 2611.1099412 天前

RoboMoCap:200小时自视角光捕数据集支撑人形机器人运动学习 ​

本文由 AI 根据公开内容整理,可能有误,请以原文为准:RoboMoCap: A Large-Scale Whole-Body Motion Capture Dataset with Egocentric Vision for Robot Learning(arXiv 2611.10994)

核心内容 ​

针对人形机器人从人类演示中学习全身协调运动时面临的自视角下肢遮挡与高保真数据缺失难题,研究团队推出RoboMoCap大规模多模态数据集。该数据集包含14名参与者超200小时录制内容,同步采集6个头戴相机、腕部视觉惯性传感及120Hz标记点光学全身动捕数据,系统覆盖变速行走、转弯、受限导航、地形穿越、物体搬运及移动中操作等多样化运动原语与交互任务,为以运动为核心的人形机器人学习提供高质量真值基础。

RoboMoCap提供时间对齐的多模态观测与度量级全身运动表征,使算法能推断自视角不可见区域(如下肢与足部)的运动状态。基于该数据集建立了三个核心基准任务:自视角全身运动重建、足部接触与落点估计、以及人类到人形机器人的动作重定向。通过将自视角视觉与高保真人体运动在多样场景下精确配对,该数据集填补了现有自视角数据集缺乏光学动捕真值及系统性运动操作覆盖的空白,直接支撑具身智能体从第一人称演示数据中进行端到端策略学习与泛化验证。

关键看点 ​

  • 数据规格:收录14人超200小时数据,集成6路头戴相机、腕部IMU及120Hz光学全身动捕,实现毫秒级多模态时间对齐
  • 任务覆盖:系统包含变速行走、转弯、地形穿越、物体搬运及移动操作等运动原语,解决自视角下肢遮挡导致的真值缺失问题
  • 基准评测:设立自视角全身运动重建、足部接触/落点估计、人形动作重定向三大Benchmark,量化评估运动理解与迁移能力
  • 核心价值:提供度量级全身运动表征与不可见肢体真值,支撑人形机器人从第一人称演示数据进行端到端全身协调策略学习

工程与产业观察 ​

该数据集精准击中人形机器人模仿学习中“自视角输入”与“全身控制输出”的真值断层痛点。其120Hz光捕与多目视觉的严格对齐,为训练下肢遮挡下的状态估计网络及Sim-to-Real动作重定向提供了稀缺的高置信度监督信号,是验证具身大模型运动泛化能力的关键基础设施。

英文摘要 (Abstract) ​

Humanoid robots must coordinate locomotion and manipulation to navigate complex environments and perform everyday tasks. Learning such behaviours from human demonstrations requires accurate whole-body motion paired with egocentric vision, particularly for lower-body movements and foot placements that are frequently invisible from a first-person perspective. However, existing egocentric datasets often lack high-fidelity optical motion capture or systematic coverage of diverse loco-manipulation behaviours. To address this gap, we introduce RoboMoCap, a large-scale multimodal dataset combining egocentric vision with marker-based optical motion capture, specifically designed for locomotion-centric humanoid learning. RoboMoCap comprises over 200 hours of recordings from 14 participants, featuring six head-mounted cameras, wrist-mounted visual and inertial sensing, and optical full-body motion capture at 120 Hz. The dataset systematically covers diverse locomotion primitives, dynamic whole-body movements, and loco-manipulation tasks, including variable-speed walking, turning, constrained navigation, terrain traversal, object transportation, and manipulation during locomotion. It provides temporally aligned multimodal observations and metric whole-body motion representations, enabling the study of human movement beyond directly observable body regions. We systematically evaluate existing methods on three benchmark tasks: egocentric full-body motion reconstruction, foot contact and placement estimation, and human-to-humanoid motion retargeting. By pairing egocentric vision with high-fidelity human motion across diverse locomotion and interaction scenarios, RoboMoCap provides a foundation for egocentric motion understanding and humanoid learning from human demonstrations.

作者:Mayur Deshmukh, Aaron Tung, Junyi Li, Nikash Bhardwaj, Ademi Adeniji, Vincent Liu, Michael Cho, Jiafei Duan

基于本页技术方向关联的整机参数、核心零部件、实测报告与同赛道选型横评。

外部来源

本页事实对应的公开页面,以原文为准。(点击将离开本站)

  1. arXiv 2611.10994:RoboMoCap: A Large-Scale Whole-Body Motion Capture Dataset with Egocentric Vision for Robot Learning ↗访问 2026-11-11