外观
论文HF Daily Papers · arXiv 2610.303207 天前
FRED框架通过自蒸馏优化WAM单步表征，零样本人类视频迁移成功率达50%
本文由 AI 根据公开内容整理，可能有误，请以原文为准：Self-Distilling Future Video Representations for World Action Models（HF Daily Papers · arXiv 2610.30320）
核心内容
针对世界动作模型（WAMs）依赖迭代去噪导致推理效率低的问题，研究提出FRED框架以优化单步去噪下的未来视频表征质量。该方法通过自蒸馏机制，以真实观测提取的表征为目标训练噪声预测，并引入历史观测提供时序上下文。实验表明，FRED在长程任务与域外泛化场景中一致优于现有WAMs，特别是在仅有人类演示视频的零样本技能迁移任务中，任务成功率达50.0%，较最优基线提升超30个百分点。
技术层面，FRED摒弃了传统WAMs对多步迭代生成的依赖，转而通过自蒸馏直接对齐单步中间层特征与Ground Truth表征，解决了动作解码器对低质中间特征的敏感问题。系统架构上，模型显式条件化历史观测序列，增强了从噪声恢复未来状态的时序一致性。这种设计使得机器人策略能直接从非结构化的人类视频中提取有效时空先验，大幅降低了具身智能对昂贵真机数据的依赖，验证了高质量视频表征作为通用策略学习基础的有效性。
关键看点
- 自蒸馏机制：以真实未来观测表征为Target，训练模型从噪声中单步预测高质量中间特征，避免迭代去噪开销
- 时序上下文增强：将历史观测序列作为条件输入，辅助模型在单步推理中更准确恢复未来状态表征
- 人类视频迁移：在无机器人训练数据场景下，仅凭人类演示视频实现50.0%任务成功率，远超基线19.4%
- 泛化性能验证：在长程操作与零样本域外环境中表现稳健，证明单步优质表征对通用策略学习的关键作用
工程与产业观察
FRED证明了在不牺牲生成质量的前提下，通过表征对齐可将视频扩散模型转化为高效的动作预测器。其核心价值在于打通了海量人类视频到机器人策略的低成本迁移路径，为具身大模型摆脱真机数据瓶颈提供了可行的工程范式，对构建通用操作基础模型具有重要参考意义。
英文摘要 (Abstract)
World Action Models (WAMs) have emerged as a promising approach to learning generalist robot policies using rich spatiotemporal priors from pre-trained video diffusion models. A recent line of WAMs improves inference efficiency by conditioning a separate action decoder on intermediate future video representations extracted from a single denoising step, avoiding costly iterative denoising required for video generation. This design makes action prediction depend directly on the quality of these future representations. In this paper, we introduce Future REpresentation learning via self-Distillation with temporal context (FRED), a framework that trains WAMs to improve the quality of future video representations from noise in a single denoising step. Specifically, FRED uses self-distillation to predict future representations from noise, using representations extracted from ground-truth future observations as targets. We further condition the video model on historical observations, providing temporal context for predicting future representations better. Across diverse real-world manipulation scenarios, FRED consistently outperforms competing WAMs. It achieves particularly large gains on long-horizon and out-of-domain tasks under zero-shot environment generalization, and substantially improves human-to-robot skill transfer, reaching 50.0% task success on tasks absent from robot training data and demonstrated only in human videos, compared with 19.4% for the best competing WAM.
Hugging Face Daily Papers 热度：37 upvotes 作者：Huiwon Jang, Suhyeok Jang, Hyeonmo Kang, Seungku Kim, Jinwoo Shin, Sihyun Yu
来源
本页事实对应的公开页面，以原文为准。