# 用物理一致交互将人类视频机器人化

> 原文链接：https://robotics.hellobike.com/papers/2026-10-05-2610-06137
> 频道：papers | 更新日期：2026-10-05 | 标签：arxiv, cs.RO
> 核心摘要：该方法将人类操作视频转为物理一致的机器人兼容视频，并在RoboTwin任务与真实部署中验证共训练效果。

# 用物理一致交互将人类视频机器人化

::: v-pre
> 原文链接：[Robotizing Human Videos with Physically Consistent Interactions](https://arxiv.org/abs/2610.06137)（arXiv 2610.06137 · 发布日期：2026-10-05）

**Robotizing Human Videos with Physically Consistent Interactions**

该方法将人类操作视频转为物理一致的机器人兼容视频，并在RoboTwin任务与真实部署中验证共训练效果。

## 核心要点

- 人类视频可扩展操作数据，但人手与机器人夹爪的具身差异限制了直接使用。
- 方法结合交互感知接触重建和深度感知合成，恢复3D接触并约束机器人-物体遮挡。
- 在四个RoboTwin任务和两种Diffusion Policy视觉编码器上，平均成功率最高。
- 真实部署显示共训练提升对视觉干扰物的鲁棒性，但深度敏感抓取受单相机设置限制。

**验证范围与局限**：验证包括四个RoboTwin任务和真实部署，且真实深度敏感抓取受单相机设置限制。

## 英文摘要 (Abstract)

Human videos offer scalable manipulation data, but the embodiment gap between human hands and robot manipulators limits their direct use. Existing video-editing methods replace hands with rendered robots, yet inaccurate interaction reconstruction and compositing can produce inconsistent grasps and implausible robot-object occlusions. We address these failures from two complementary physical aspects: interaction geometry and scene visibility. First, an interaction-aware contact reconstruction module combines hand-object segmentation with mesh-level contact prediction to recover dense 3D contacts, then converts them into temporally stabilized grasps for parallel-jaw grippers. Second, a depth-aware compositing module uses scene and robot depth to enforce physically consistent robot-object occlusions. The resulting videos preserve the interaction structure of human demonstrations in a robot-compatible form and are co-trained with robot demonstrations. Using identical human videos and robot data, we compare against robot-only training and the original Masquerade pipeline. Across four RoboTwin tasks and two Diffusion Policy visual encoders, our method achieves the highest average success rates, with especially strong gains under out-of-distribution scene variation. Real-world deployment further shows that the proposed co-training approach improves robustness to visual distractors when the task geometry is observable, while performance on depth-sensitive grasps remains limited by the single-camera setup.

作者：Ching-Lam Cheng, Shengfeng He, Bin Zhu
:::

## 原始参考来源 (Sources)

- [arXiv 2610.06137：Robotizing Human Videos with Physically Consistent Interactions](https://arxiv.org/abs/2610.06137)