外观
论文arXiv 2610.1456118 天前
从视频到视觉：从人类运动中学习可泛化的人形机器人搬运
原文链接：From Video to Vision: Learning Generalizable Humanoid Carrying from Human Motion（arXiv 2610.14561 · 发布日期：2026-10-10）
From Video to Vision: Learning Generalizable Humanoid Carrying from Human Motion
MOVER针对人形机器人搬运空间泛化差的问题，以视频重定向、多阶段RL与蒸馏训练，在仿真和真机搬运中验证。
核心要点
- 现有方法依赖实时参考轨迹和动捕系统，物体只能从指定位置搬运，空间泛化较差。
- 方法从视频采集人类运动，用GENMO和GMR重定向，并通过数据增强覆盖不同形状和尺寸物体。
- 多阶段Teacher-Assistant-Student RL先学轨迹搬运，再放松跟踪并用Dagger-PPO蒸馏去除全局位置信息依赖。
- 仿真实验比较阶段间及蒸馏前后成功率，并将策略迁移到真机实现无动捕搬运。
验证范围与局限：验证包括仿真成功率对比，以及在真实人形机器人上进行不同位置物体的无动捕搬运。
英文摘要 (Abstract)
Most existing approaches rely on video or motion-capture data for reinforcement learning (RL) training; however, all of these methods require an upper-level module to generate reference trajectories in real time. In object carrying tasks, the humanoid can only lift objects from designated locations along the reference trajectory, resulting in poor spatial generalization. Moreover, existing policies can only be evaluated inside a motion-capture studio. To address these limitations, we propose \textbf{MOVER}, a multi-stage, Teacher-Assistant-Student, RL training framework for humanoid object carrying. We first collect data from videos and employ GENMO and GMR to retarget the video data, combining this with data augmentation techniques to generalize the data across objects of varying shapes and sizes. In the \textbf{first stage}, building upon the DeepMimic \cite{peng2018deepmimic} motion-training framework, we train the humanoid to transport objects from specified locations along designated trajectories. In the \textbf{second stage}, we relax the tracking of the reference motion and adjust the reward so that the humanoid learns to transport objects from different positions. In the \textbf{third stage}, we apply a Dagger-PPO distillation method to distill away the proprioceptive position observations and reference joint position of the humanoid. This allows the deployed policy to operate without relying on global position information from a motion-capture system; instead, the humanoid can directly perceive the object's pose from its onboard RGB-D camera to complete the carrying task. We design a series of simulation experiments to compare the success rates across stages (Stages 1 vs. 2) and before versus after distillation. We further transfer the policy to a real humanoid, successfully transporting objects at varying positions in a motion-capture-free setting.
作者：Ziluo Ding
来源
本页事实对应的公开页面，以原文为准。