外观
论文arXiv 2610.1373219 天前
LAPT-Scale：通过人类视频上的潜在动作预训练扩展具身策略
原文链接：LAPT-Scale: Scaling Embodied Policies with Latent Action Pre-Training on Human Videos（arXiv 2610.13732 · 发布日期：2026-10-09）
LAPT-Scale: Scaling Embodied Policies with Latent Action Pre-Training on Human Videos
LAPT-Scale用人类第一视角视频的连续潜在动作预训练VLA策略，并在真机上验证成功率提升与视觉泛化。
核心要点
- 通用机器人策略扩展受具身交互数据采集成本限制，人类视频缺少动作标签。
- LAPT-Scale-LAM从帧间视觉转移学习连续潜在动作，保留细粒度运动与交互动态。
- LAPT-Scale-Policy结合人类视频预训练、目标机器人具身对齐和任务后训练。
- 标注视频1K至5K小时，成功率63.3%升至85.0%，并泛化到未见视觉条件。
验证范围与局限：验证范围包括真机成功率和未见视觉条件泛化，摘要未说明机器人数量或具体任务。
英文摘要 (Abstract)
Scaling generalist robotic policies is limited by the high cost of collecting embodied interaction data. Large-scale egocentric human videos offer a rich alternative, but they lack robot action labels, and existing action annotation methods are often noisy under real-world visual complexity and diverse manipulation dynamics. We introduce LAPT-Scale, a scalable latent action pre-training paradigm with two components: LAPT-Scale-LAM learns continuous latent actions by explaining frame-to-frame visual transitions in human videos and uses them as human-derived supervision, while LAPT-Scale-Policy trains a VLA policy through human-video latent-action pre-training, target-robot embodiment alignment, and task-specific post-training. Unlike prior vector-quantized LAMs, LAPT-Scale-LAM uses a continuous latent space to preserve fine-grained motion and interaction dynamics. We further combine perceptual, semantic, and optical-flow alignment signals to improve transition-level supervision beyond pixel reconstruction. Trained on 23.3K hours of egocentric video, LAPT-Scale-LAM provides implicit action supervision for policy pre-training, yielding consistent scaling gains and improving real-robot success from 63.3% to 85.0% as annotated human video scales from 1K to 5K hours. The learned policy also generalizes to unseen visual conditions.
作者：Zhiyuan Xiang, Peidong Liu, Mingyang Li, Jiawei Li, Wenhow Li, Wenbo Ding, Xiang Deng
来源
本页事实对应的公开页面，以原文为准。