# 从单目视频进行保持几何的人到机器人上半身动作重定向

> 原文链接：https://robotics.hellobike.com/papers/2026-09-29-2609-37776
> 频道：papers | 更新日期：2026-09-29 | 标签：arxiv, cs.RO
> 核心摘要：提出保持几何的人到机器人上身动作重定向框架，从单目视频重建并映射动作，在16段视频、仿真和代表性真机演示中验证。

# 从单目视频进行保持几何的人到机器人上半身动作重定向

::: v-pre
> 原文链接：[Geometry-Preserving Human-to-Robot Upper-Body Motion Retargeting from Monocular Video](https://arxiv.org/abs/2609.37776)（arXiv 2609.37776 · 发布日期：2026-09-29）

**Geometry-Preserving Human-to-Robot Upper-Body Motion Retargeting from Monocular Video**

提出保持几何的人到机器人上身动作重定向框架，从单目视频重建并映射动作，在16段视频、仿真和代表性真机演示中验证。

## 核心要点

- 单目RGB视频易获取，但身手尺度、运动学差异和末端细节保持带来挑战。
- 框架联合帧级身体、视频级观测与手部证据，约束可微MHR状态并修复手部伪影。
- 以臂段方向、肘部构型、掌姿态和双腕关系表示动作，经多阶段逆运动学实现。
- 16段视频上手部重投影误差由22.36降至7.21像素，轨迹均完成仿真回放。

**验证范围与局限**：验证范围为16段单目视频、运动学仿真回放，以及代表性签名和抓取动作的真机演示。

## 英文摘要 (Abstract)

Monocular RGB video provides an accessible source of human demonstrations for upper-body robot motion, yet video-driven human-to-robot transfer remains challenging because body and hand motion are recovered at different spatial scales, human and robot kinematics differ substantially, and fine distal motion is difficult to preserve across embodiments. We present a geometry-preserving motion-retargeting framework that integrates unified body--hand reconstruction with morphology-independent geometric transfer. Frame-wise body estimates, video-level observations, and detailed hand evidence jointly constrain a single differentiable Momentum Human Rig (MHR) state, while transient hand artifacts are repaired in parameter space. The reconstructed motion is represented by arm-segment directions, elbow configuration, relative palm orientation, and bilateral wrist relations, and is realized on the target robot through multi-stage inverse kinematics and robot-specific hand adaptation. Within the broader system, Across-VAM provides video generation, whereas Across-WAM performs human-to-robot motion mapping. The method is evaluated on 16 monocular videos comprising 1,769 source frames, including 10 signing and six reach-to-grasp sequences. Unified reconstruction reduces mean hand reprojection error from 22.36 to 7.21 pixels relative to SAM 3D Body. All 16 retargeted trajectories completed kinematic simulation playback, and representative signing and reach-to-grasp motions were further demonstrated on a physical robot. The results demonstrate a unified pipeline from monocular human video to coordinated upper-body motion on a dual-arm dexterous robot.

作者：Xiaoyu Yang, Sen Han, Da Li, Nan Wu
:::

## 原始参考来源 (Sources)

- [arXiv 2609.37776：Geometry-Preserving Human-to-Robot Upper-Body Motion Retargeting from Monocular Video](https://arxiv.org/abs/2609.37776)