外观
论文arXiv 2610.1625616 天前
EgoWholeView：仅用相机的头戴式身体与双手运动捕捉用于具身示教
原文链接：EgoWholeView: Camera-Only Head-Mounted Capture of Body and Bimanual Hand Motion for Embodied Demonstrations（arXiv 2610.16256 · 发布日期：2026-10-12）
EgoWholeView: Camera-Only Head-Mounted Capture of Body and Bimanual Hand Motion for Embodied Demonstrations
EgoWholeView用六个同步广角RGB头戴相机与专用融合流程，捕捉身体和双手运动，并在六相机记录上验证。
核心要点
- 单目自我视角相机受视野有限、尺度歧义和手物遮挡影响。
- 系统用外向视角定位装置，下视重叠相机分别预测SMPL-X身体和MANO手部参数。
- 基于预测可靠性融合身体与手部关节旋转，并跨视角和帧聚合形状。
- 相对单目预测，融合降低身体和手部PA-MPJPE；时间细化降低角加速度幅值。
验证范围与局限：验证范围为头戴六个RGB相机的记录及相对单目预测比较，且不使用第三人称视角。
英文摘要 (Abstract)
Head-mounted body and bimanual motion capture supports self-avatars, teleoperation, and embodied demonstrations. However, a single egocentric camera is vulnerable to a limited field of view, scale ambiguity, and hand-object occlusion. We present EgoWholeView, a wearable capture system of six synchronized wide-angle RGB cameras with a role-specialized pipeline. The outward views localize the rig in a world frame, while an overlapping downward pair independently predicts SMPL-X body and MANO hand parameters. Both predictors are adapted to egocentric fisheye data. To combine the two, reliability cues measured from the predictions themselves weight the fusion of body and hand joint rotations, while shape is aggregated across views and frames. The pair's baseline fixes metric scale, and triangulation recovers root translation. Relative to monocular prediction, fusion reduces PA-MPJPE by 3.7% for the body and 11.2% for the hands, and temporal refinement lowers angular-acceleration magnitude by 32.2% and 31.0%, respectively. On our six-camera recording, the pipeline produces body and hand states in a shared world frame using benchmark-fitted gates without retuning, and temporal refinement lowers hand acceleration magnitude by 41.08%. Using only RGB cameras and no third-person view, EgoWholeView offers a low-cost, portable route to egocentric interaction data.
作者：Yaoqing Hu, Xuguo He, Daohan Cui, Yukun Shi, Kai Chen
来源
本页事实对应的公开页面，以原文为准。