# Ego2Sim2Humanoid：无机器人示教实现G1移动操作VLA训练

> 原文链接：https://robotics.hellobike.com/papers/2026-11-17-2611-19196-7e206922
> 频道：papers | 更新日期：2026-11-17 | 标签：auto, arxiv
> 核心摘要：提出Ego2Sim2Humanoid框架，将第一人称眼镜视频转化为物理验证的机器人轨迹。无需真机遥操示教，仅凭转换数据微调GR00T N1.7模型，即在Unitree G1上实现自主移动操作，解决人形机器人模仿学习数据采集瓶颈。

# Ego2Sim2Humanoid：无机器人示教实现G1移动操作VLA训练

::: v-pre
> 本文由 AI 根据公开内容整理，可能有误，请以原文为准：[Ego2Sim2Humanoid: Agentic Real-to-Sim for Learning Loco-Manipulation VLAs without Robot Demonstrations](https://arxiv.org/abs/2611.19196)（arXiv 2611.19196）

## 核心内容

针对人形机器人模仿学习依赖昂贵真机遥操示教的痛点，arXiv论文提出Ego2Sim2Humanoid方法。该框架利用第一人称眼镜采集人类演示，通过Agentic Real-to-Sim阶段重建带碰撞与纹理的视觉资产并预测接触带，再经运动适配与两阶段修正生成物理可行的机器人轨迹。最终仅使用此合成数据微调GR00T N1.7模型，成功在Unitree G1上实现零真机示教的自主移动操作任务。

技术链路包含抓取规划器将接触带转为腕部姿态、双层站立优化保持手部可达、全身运动生成器合成下肢步态及视野外臂部动作。强化学习跟踪策略在物理仿真中基于语义奖励训练，确保物体离开视野时仍能完成操作。系统同步渲染机器人视角图像、本体感知与验证动作标签，构建了从人类自然演示到具身智能体策略学习的闭环数据管线，规避了穿戴设备与手持夹具的形态差异限制。

## 关键看点

- **数据源替代**：直接使用第一人称眼镜RGB-D流重建场景，无需机器人本体或手持夹具参与采集，大幅降低示教成本
- **运动重定向**：采用双层站立优化与全身运动生成器，解决人与机器人肢体尺寸差异导致的接触不可行问题
- **仿真验证闭环**：在物理引擎中训练RL跟踪策略并渲染同步多模态数据，确保合成轨迹具备物理真实性与可执行性
- **零真机示教落地**：仅依赖转换数据集微调GR00T N1.7，在Unitree G1上验证了移动操作任务的自主执行能力

## 工程与产业观察

该工作验证了“人类视频→仿真资产→策略训练”管线的可行性，为人形机器人提供了低成本数据飞轮范式。通过将非结构化人类演示转化为物理合规的仿真标签，有效缓解了具身大模型对高质量真机数据的依赖，对推动通用移动操作算法的快速迭代具有重要工程参考价值。

## 英文摘要 (Abstract)

Imitation learning for visuomotor policies typically relies on teleoperated robot demonstrations, which are costly because every episode requires a trained operator controlling a physical robot in real time through teleoperation interfaces and wearable body-tracking hardware. Egocentric glasses capture human demonstrations without a robot or handheld grippers, yet they track only the head and in-view human hands, and directly copying human motion to a robot with different stature, arm reach and hand morphology fails to produce feasible contacts. \method{} converts egocentric glasses recordings into physically verified robot training trajectories. An agentic real-to-sim stage reconstructs collision and textured visual assets from the recorded RGB-D and head streams, completes unseen surfaces when needed, and predicts target contact bands on the manipulated objects. During motion adaptation, a grasp planner turns each contact band into a feasible robot wrist pose and finger closure, bilevel stance optimization plans collision-free standing postures that keep hand targets reachable, and a whole-body motion generator synthesizes lower-body locomotion and out-of-view arm motion. Two-stage motion correction then blends visibility transitions kinematically and trains a reinforcement-learning tracking policy in physics simulation under agent-designed semantic rewards, completing manipulation phases even when the object leaves the camera view. Rendering the reconstructed scene through the robot's onboard camera along the optimized rollout yields synchronized robot-view images, proprioception and verified action labels. Fine-tuned exclusively on this converted dataset, without real-robot demonstrations, GR00T N1.7 performed loco-manipulation tasks autonomously on a Unitree G1. Project page: https://robonana.github.io/ego2sim2humanoid/

作者：Chen Liu, Nikolaus Correll
:::

## 原始参考来源 (Sources)

- [arXiv 2611.19196：Ego2Sim2Humanoid: Agentic Real-to-Sim for Learning Loco-Manipulation VLAs without Robot Demonstrations](https://arxiv.org/abs/2611.19196)