外观
论文arXiv 2609.170991 个月前
GeoLAM：从无标签人类视频中学习几何扎根的潜动作
原文链接：GeoLAM: Learning Geometry-Grounded Latent Actions from Unlabeled Human Videos（arXiv 2609.17099 · 发布日期：2026-09-15）
GeoLAM: Learning Geometry-Grounded Latent Actions from Unlabeled Human Videos
GeoLAM用冻结几何特征层级和训练期4D几何教师，从无动作标签人类视频学习潜动作，并在基准与机器人操作任务上验证。
核心要点
- 人类视频蕴含丰富操作经验，但视觉重建易混淆操作运动、外观变化和相机运动。
- GeoLAM结合冻结几何特征层级的未来帧重建，以及训练期4D几何教师的运动监督。
- 教师目标捕获3D位移、残余图像平面运动和表面朝向变化，并用可见性与置信度加权降低不可靠估计影响。
- 预训练后表示用于带动作标签机器人示教的世界-动作模型，评估显示GeoLAM在潜动作基准和机器人操作任务上表现强劲。
验证范围与局限：验证范围为摘要所述的潜动作基准和机器人操作任务，未说明具体仿真或真机设置。
英文摘要 (Abstract)
Human videos provide rich manipulation experience, but extracting action representations that preserve useful motion remains challenging. Visual reconstruction alone can entangle manipulation-related motion with appearance changes and camera movement. We present GeoLAM, a framework for learning geometry-grounded latent actions from action-free human videos. GeoLAM combines future-frame reconstruction through a frozen geometric feature hierarchy with motion supervision from a training-only 4D geometry teacher. The geometric representation provides a structural prior, while the teacher's predictions yield spatially pooled targets capturing 3D displacement, residual image-plane motion, and surface-orientation changes. Visibility and confidence weighting reduces the contribution of unreliable estimates, encouraging continuous latent actions to retain geometric motion without explicit hand-pose or hand-trajectory annotations. After video pretraining without action labels, the learned representation provides transition targets for a world-action model trained on action-labeled robot demonstrations. The model jointly denoises latent actions and executable action chunks, with future-video prediction used only as an auxiliary training task. Deployment therefore requires neither the geometry teacher nor future-video generation. Evaluations on a latent-action benchmark and robotic manipulation tasks demonstrate the strong performance of GeoLAM.
作者：Yifan Xie, Hekun Tian, Jinkun Liu, YuAn Wang, Qiao Sun, Wenbo Ding
来源
本页事实对应的公开页面，以原文为准。