# EgoX框架通过具身梦境预训练提升跨机体操作泛化性能

> 原文链接：https://robotics.hellobike.com/papers/2026-11-12-2611-16728-3966540c
> 频道：papers | 更新日期：2026-11-12 | 标签：auto, arxiv, hf-daily-papers
> 核心摘要：EgoX提出模块化跨具身Transformer架构MXT+，利用具身梦境目标在异构数据上预训练。在6种机器人13个任务对中，平均归一化进度从0.63升至0.80，分布外场景从0.53提升至0.77，验证了跨形态学习有效性。

# EgoX框架通过具身梦境预训练提升跨机体操作泛化性能

::: v-pre
> 本文由 AI 根据公开内容整理，可能有误，请以原文为准：[EgoX: Egocentric Cross-Embodiment Manipulation with Embodiment Dreaming](https://arxiv.org/abs/2611.16728)（HF Daily Papers · arXiv 2611.16728）

## 核心内容

针对人类、真机与仿真智能体因形态与控制接口差异导致的跨具身学习难题，研究团队提出EgoX第一人称跨具身操作学习框架。该框架通过语义对齐的观测与动作模态组织异构演示数据，并在包含5种物理机器人与1种仿真人形机器人的6种本体、5类任务共13个任务-本体对上进行验证。实验表明，经跨具身预训练的MXT+模型平均归一化任务进度达0.80，显著优于从头训练的0.63。

技术上，MXT+为每种观测模态分配专用Tokenizer、每种动作模态分配专用Expert，并共享Transformer主干网络。核心创新在于引入仅用于训练的“具身梦境”目标，预测未来视觉与本体感知状态的潜在表示，迫使共享主干建模各本体动力学特性。消融实验证实，更广泛的预训练混合数据、模块化接口及梦境机制均带来收益，其中梦境机制有效促进了不同本体间潜在空间几何结构的一致性，使分布外任务进度由0.53跃升至0.77。

## 关键看点

- **跨具身架构**：MXT+采用模态专用Tokenizer/Expert与共享Trunk设计，解决异构本体感知控制接口不兼容问题
- **具身梦境机制**：通过预测未来视觉与本体状态潜在表示，强制共享主干学习各本体动力学，统一潜在空间几何
- **量化性能提升**：在6种机器人13个任务对上，预训练使平均归一化进度从0.63升至0.80，OOD场景从0.53升至0.77
- **异构数据融合**：整合人类、真机与仿真智能体演示，通过语义对应模态组织数据，验证广泛预训练混合数据的增益

## 工程与产业观察

EgoX证明了在不统一硬件前提下，通过架构解耦与生成式辅助目标可有效挖掘异构具身数据价值。其模块化接口设计与梦境预训练范式，为缓解具身智能数据孤岛、提升策略跨本体迁移能力提供了可复现的工程路径，对构建通用操作基础模型具有重要参考意义。

## 英文摘要 (Abstract)

Generalist manipulation policies can benefit from the complementary experience of humans, real robots, and simulated agents. However, differences in morphology, sensing, and control interfaces make it difficult to learn effectively from their combined demonstrations. We introduce EgoX, an egocentric cross-embodiment learning framework that organizes heterogeneous demonstrations through semantically corresponding observation and action modalities. EgoX pretrains a Modularized Cross-Embodiment Transformer with Embodiment Dreaming (MXT+) on this mixture and fine-tunes it on target-embodiment demonstrations. Within each embodiment, MXT+ assigns a dedicated tokenizer to each observation modality and a dedicated expert to each action modality, with a Transformer trunk shared across embodiments. A training-only embodiment-dreaming objective predicts latent representations of future visual and proprioceptive states, encouraging the shared trunk to model each embodiment's dynamics. Across 13 task-embodiment pairs spanning five tasks and six robot embodiments, including five physical robots and one simulated humanoid, cross-embodiment pretraining raises MXT+'s average normalized task progress from 0.63 to 0.80 over training from scratch. The gains are particularly pronounced under out-of-distribution conditions, where average task progress increases from 0.53 to 0.77. Ablations demonstrate the benefits of broader pretraining mixtures, modular interfaces, and embodiment dreaming, with dreaming producing more similar latent geometry across embodiments. Project page: egox-dream.github.io.

Hugging Face Daily Papers 热度：1 upvotes
作者：Yaru Niu, Binghong Chen, Changwei Yao, Shangtao Li, Zhenlong Fang, Revanth Krishna Senthilkumaran, Shuai Zhou, Yuemin Mao, Kalie Ching, Po-Ju Chao, Yudong Luo, Peiqi Yu, Heng Zhang, Hao Zhang, Bingqin…
:::

## 原始参考来源 (Sources)

- [HF Daily Papers · arXiv 2611.16728：EgoX: Egocentric Cross-Embodiment Manipulation with Embodiment Dreaming](https://arxiv.org/abs/2611.16728)