外观
论文HF Daily Papers · arXiv 2610.37219昨天
JAM框架解耦VLA与WAM：推理零生成且训练FLOPs降73%
本文由 AI 根据公开内容整理，可能有误，请以原文为准：Vision-Language-Action or World-Action? Just Action Models（HF Daily Papers · arXiv 2610.37219）
核心内容
针对VLA与WAM技术路线之争，研究指出二者差异源于骨干初始化、未来观测目标及推理机制的耦合。新提出的JAM框架将这三者解耦，采用预训练视觉骨干加动作专家架构，仅依赖动作损失与前瞻损失训练。该方法在LIBERO、LIBERO-Plus及RoboTwin基准测试中全面超越或持平现有WAM模型，证明未来监督信号可独立于视觉生成存在。
JAM核心机制是利用预测头训练当前特征去匹配冻结的未来编码，该预测头在推理时直接丢弃，彻底消除视觉生成开销。实验显示，相比FastWAM，JAM在LIBERO三项任务成功率分别提升至98.2%、64.5%和91.4%；借助缓存前瞻目标，单次更新训练FLOPs减少73%。结果表明无论使用生成器初始化还是VLM骨干，均可在不增加推理负担前提下获得强前瞻性能。
关键看点
- 架构解耦：JAM将骨干选择、未来目标与推理机制分离，推理阶段完全移除视觉生成模块，仅保留动作输出通路
- 性能提升：在LIBERO基准上成功率从FastWAM的97.0%/55.3%/91.2%提升至98.2%/64.5%/91.4%，RoboTwin表现持平或略优
- 训练提效：通过缓存前瞻目标策略，JAM相比FastWAM减少73%的在线训练FLOPs，显著降低算力成本
- 骨干通用：验证了前瞻损失对VLM或生成器初始化骨干均有效，打破特定预训练模型对控制性能的垄断
工程与产业观察
JAM为具身大模型训练提供轻量化新范式：通过将“理解未来”转化为特征空间的回归任务而非像素生成，既保留了世界模型的时序预测优势，又规避了高昂的推理计算。这对端侧部署及高频控制场景极具工程价值，表明算法设计应优先解耦表征学习与生成负载。
英文摘要 (Abstract)
The vision-language-action (VLA) and world-action model (WAM) communities debate whether control policies should inherit pretrained knowledge from vision-language models or from visual generators. The debate, however, bundles three separable choices: backbone initialization, the objective applied to future observations, and whether the machinery for that future loss is kept at inference. A gap between a VLA and a WAM therefore cannot simply be attributed to any one of them. We show that decoupling these choices yields a common framework for both families, which we term Just Action Model (JAM): a pretrained visual backbone with an action expert, trained with an action loss and a foresight loss. The foresight loss trains current-observation features to predict frozen encodings of the recorded future through a prediction head discarded after training, so no visual generation is involved in either training or inference, and the backbone can be chosen freely. We first compare objectives within fixed policy architectures, then evaluate the foresight recipe across backbone families. The resulting JAM policies outperform their matched FastWAM and ImageWAM counterparts on LIBERO and LIBERO-Plus, and achieve similar or slightly higher success rates on RoboTwin. Compared with FastWAM, JAM improves the success rates from 97.0%/55.3%/91.2% to 98.2%/64.5%/91.4%, respectively. With cached foresight targets, JAM reduces estimated online training FLOPs per sample per update by 73% relative to FastWAM. Our results show that future supervision improves control without visual generation or additional inference machinery, and strong foresight performance is attainable with either generator-initialized or vision-language backbones. Project Page: https://y0uroy.github.io/JAM
Hugging Face Daily Papers 热度：3 upvotes 作者：Yonghao Yu, Lang Huang, Runyi Li, Toshihiko Yamasaki
来源
本页事实对应的公开页面，以原文为准。