# EgoJEPA：超越重定向的从第一视角视频学习灵巧操作

> 原文链接：https://robotics.hellobike.com/papers/2026-10-14-2610-20904
> 频道：papers | 更新日期：2026-10-14 | 标签：arxiv, cs.RO
> 核心摘要：EgoJEPA为人类视频到机器人控制迁移去除规则重定向，以三阶段表征与世界建模在真实操作任务上验证。

# EgoJEPA：超越重定向的从第一视角视频学习灵巧操作

::: v-pre
> 原文链接：[EgoJEPA: Beyond Retargeting for Dexterous Manipulation from Egocentric Videos](https://arxiv.org/abs/2610.20904)（arXiv 2610.20904 · 发布日期：2026-10-14）

**EgoJEPA: Beyond Retargeting for Dexterous Manipulation from Egocentric Videos**

EgoJEPA为人类视频到机器人控制迁移去除规则重定向，以三阶段表征与世界建模在真实操作任务上验证。

## 核心要点

- 第一视角人类视频可扩展，但几何重定向需反复校准且会传递手跟踪误差。
- 框架以JEPA式视觉预测预训练效应感知潜动作，再进行潜动作世界建模。
- 预训练与机器人控制参数化解耦，并用轻量动作适配器适配不同机器人。
- 在五个真实操作任务、三种机器人形态上，域外成功率74%，高于65%基线且耗时更低。

**验证范围与局限**：验证范围为五个真实世界操作任务，覆盖三种机器人形态。

## 英文摘要 (Abstract)

Egocentric human videos offer a scalable alternative to costly robot demonstra- tions, yet transferring their manipulation priors to robot control remains chal- lenging. The commonly adopted retargeting-based approaches convert human wrist and finger poses into embodiment-specific action commands through hand- crafted geometric mappings. However, such mappings require meticulous recal- ibration across different embodiments and risk propagating hand-tracking errors into downstream robot control pipelines. To this end, we propose EgoJEPA, a three-stage framework that learns embodiment-agnostic representations from ego- centric human videos without rule-based human-to-robot retargeting: 1) Human- action pre-training learns effect-aware latent actions through JEPA-style visual prediction conditioned on hand-motion differences; 2) Latent-action-based world modeling jointly predicts future visual dynamics and latent actions, which decou- ples human prior pre-training from robot-specific control parameterizations; 3) Embodiment-specific tuning adapts the pretrained model to diverse robots through lightweight action adapters. Experiments on five real-world manipulation tasks across three robot embodiments demonstrate the effectiveness and efficiency of EgoJEPA. It achieves a 74% average success rate under the out-of-domain evalu- ation, compared with 65% for the retargeting baseline, while reducing cumulative cross-embodiment wall-clock cost by 47.2%, from 743.2 to 392.7 hours.

作者：Meng Cao, Can Zhao, Jinjie Wei, Haonan Zhao, Yulong Zhang, Duo Han, Xiaofan Li, Ziwu Song, Rongtao Xu, Ruyi Gan
:::

## 原始参考来源 (Sources)

- [arXiv 2610.20904：EgoJEPA: Beyond Retargeting for Dexterous Manipulation from Egocentric Videos](https://arxiv.org/abs/2610.20904)