外观
论文arXiv 2610.1664116 天前
什么限制了 World Action Models 的跨任务泛化？
原文链接：What Limits Cross-Task Generalization in World Action Models?（arXiv 2610.16641 · 发布日期：2026-10-12）
What Limits Cross-Task Generalization in World Action Models?
该文用AoE解耦并诊断WAM跨任务泛化，在两种世界模型和三项操作基准上以VITA提升生成未来的执行成功率。
核心要点
- 问题：WAM即使预测出合理操作未来，跨任务泛化仍然困难。
- 方法：将WAM解耦为世界模型和IDM，固定IDM并替换未来图像来源。
- 诊断：失败集中在夹爪和交互几何，生成能力耗在任务无关背景上。
- 结果：VITA使生成未来执行成功率由近零升至47.0%，并提升WAM原生控制。
验证范围与局限：验证限于摘要所述的两种世界模型、三项操作基准及WAM原生控制，未说明是否真机。
英文摘要 (Abstract)
Cross-task generalization remains difficult for world action models (WAMs), even when their predicted futures depict plausible manipulation. This gap can be attributed directly: we make this gap directly testable by decoupling a WAM into a world model and an inverse-dynamics model (IDM), then holding the IDM fixed while replacing the source of future images. We introduce Align Once, Evaluate Everywhere (AoE), a camera-space inverse-dynamics model pretrained on large-scale robot data, which provides this recovery interface for different future sources across tasks and environments. Through it, we find that actions recovered from recorded futures of unseen tasks execute almost as well as replayed demonstrations, whereas futures predicted by world models collapse across benchmarks. The failures are not generic: they concentrate on the gripper and interaction geometry while generation capacity is spent on task-irrelevant background. This diagnosis suggests that supervision should target exactly these regions. We therefore introduce the vision-to-action (VITA) loss, which passes generated futures through the frozen AoE and penalizes discrepancies in the recovered actions, redirecting generation toward action-relevant detail. Across two world models and three manipulation benchmarks, this simple change raises the execution success of generated futures from near zero to 47.0%. The same loss also transfers to WAMs and yields consistent gains in native control.
作者：Chen Zhao, Zipeng Ye, Yihan Lin, Yuchen Xie, Jing Zhang
来源
本页事实对应的公开页面，以原文为准。