Skip to content

论文arXiv 2611.191965 天前

Ego2Sim2Humanoid:无机器人示教实现G1移动操作VLA训练 ​

本文由 AI 根据公开内容整理,可能有误,请以原文为准:Ego2Sim2Humanoid: Agentic Real-to-Sim for Learning Loco-Manipulation VLAs without Robot Demonstrations(arXiv 2611.19196)

核心内容 ​

针对人形机器人模仿学习依赖昂贵真机遥操示教的痛点,arXiv论文提出Ego2Sim2Humanoid方法。该框架利用第一人称眼镜采集人类演示,通过Agentic Real-to-Sim阶段重建带碰撞与纹理的视觉资产并预测接触带,再经运动适配与两阶段修正生成物理可行的机器人轨迹。最终仅使用此合成数据微调GR00T N1.7模型,成功在Unitree G1上实现零真机示教的自主移动操作任务。

技术链路包含抓取规划器将接触带转为腕部姿态、双层站立优化保持手部可达、全身运动生成器合成下肢步态及视野外臂部动作。强化学习跟踪策略在物理仿真中基于语义奖励训练,确保物体离开视野时仍能完成操作。系统同步渲染机器人视角图像、本体感知与验证动作标签,构建了从人类自然演示到具身智能体策略学习的闭环数据管线,规避了穿戴设备与手持夹具的形态差异限制。

关键看点 ​

  • 数据源替代:直接使用第一人称眼镜RGB-D流重建场景,无需机器人本体或手持夹具参与采集,大幅降低示教成本
  • 运动重定向:采用双层站立优化与全身运动生成器,解决人与机器人肢体尺寸差异导致的接触不可行问题
  • 仿真验证闭环:在物理引擎中训练RL跟踪策略并渲染同步多模态数据,确保合成轨迹具备物理真实性与可执行性
  • 零真机示教落地:仅依赖转换数据集微调GR00T N1.7,在Unitree G1上验证了移动操作任务的自主执行能力

工程与产业观察 ​

该工作验证了“人类视频→仿真资产→策略训练”管线的可行性,为人形机器人提供了低成本数据飞轮范式。通过将非结构化人类演示转化为物理合规的仿真标签,有效缓解了具身大模型对高质量真机数据的依赖,对推动通用移动操作算法的快速迭代具有重要工程参考价值。

英文摘要 (Abstract) ​

Imitation learning for visuomotor policies typically relies on teleoperated robot demonstrations, which are costly because every episode requires a trained operator controlling a physical robot in real time through teleoperation interfaces and wearable body-tracking hardware. Egocentric glasses capture human demonstrations without a robot or handheld grippers, yet they track only the head and in-view human hands, and directly copying human motion to a robot with different stature, arm reach and hand morphology fails to produce feasible contacts. \method{} converts egocentric glasses recordings into physically verified robot training trajectories. An agentic real-to-sim stage reconstructs collision and textured visual assets from the recorded RGB-D and head streams, completes unseen surfaces when needed, and predicts target contact bands on the manipulated objects. During motion adaptation, a grasp planner turns each contact band into a feasible robot wrist pose and finger closure, bilevel stance optimization plans collision-free standing postures that keep hand targets reachable, and a whole-body motion generator synthesizes lower-body locomotion and out-of-view arm motion. Two-stage motion correction then blends visibility transitions kinematically and trains a reinforcement-learning tracking policy in physics simulation under agent-designed semantic rewards, completing manipulation phases even when the object leaves the camera view. Rendering the reconstructed scene through the robot's onboard camera along the optimized rollout yields synchronized robot-view images, proprioception and verified action labels. Fine-tuned exclusively on this converted dataset, without real-robot demonstrations, GR00T N1.7 performed loco-manipulation tasks autonomously on a Unitree G1. Project page: https://robonana.github.io/ego2sim2humanoid/

作者:Chen Liu, Nikolaus Correll

基于本页技术方向关联的整机参数、核心零部件、实测报告与同赛道选型横评。

外部来源

本页事实对应的公开页面,以原文为准。(点击将离开本站)

  1. arXiv 2611.19196:Ego2Sim2Humanoid: Agentic Real-to-Sim for Learning Loco-Manipulation VLAs without Robot Demonstrations ↗访问 2026-11-18