论文/arXiv 2610.12818·20 天前

LeWAM：基于 JEPAs 的端到端世界与动作建模

LeWAM用联合潜在状态预测与动作流匹配端到端建模世界和动作，在视觉规划及仿真和真机行为克隆任务上验证。

AI 整理

• 本体 SDK 与控制
• 仿真与训练
• 具身模型

原文链接：LeWAM: End-to-End World and Action Modeling with JEPAs（arXiv 2610.12818 · 发布日期：2026-10-08）

LeWAM: End-to-End World and Action Modeling with JEPAs

LeWAM用联合潜在状态预测与动作流匹配端到端建模世界和动作，在视觉规划及仿真和真机行为克隆任务上验证。

核心要点

• 现有 JEPA 流水线通常先训练视觉表征和动作条件动力学，再加入控制策略或求解器。
• LeWAM联合训练下一潜在状态预测和动作流匹配，并对潜在状态施加各向同性高斯约束。
• 单一模型可直接执行策略，也可用学习到的潜在动力学评估采样动作以高效规划。
• 相较既有视觉规划 JEPA，接触丰富任务成功率从28.6%升至89.7%，长时程任务从10.9%升至46.2%，规划快32.3倍。

验证范围与局限：验证包括视觉规划中的接触丰富和长时程任务，以及仿真和物理机器人上的行为克隆任务。

英文摘要 (Abstract)

Joint-embedding predictive architectures (JEPAs) learn compact latent world models without pixel reconstruction. Existing JEPA pipelines typically train visual representations and action-conditioned dynamics before adding a control policy or solver. We introduce LeWAM, an end-to-end JEPA for control with: (1) a joint training objective over next latent state prediction and action flow matching, and (2) an isotropic Gaussian constraint on the latent state, enabling a single model to support direct policy execution and efficient planning by evaluating sampled actions with its learned latent dynamics. Compared with prior JEPAs in visual planning, LeWAM improves success rate from 28.6% to 89.7% on contact-rich tasks and from 10.9% to 46.2% on long-horizon tasks, and plans 32.3 times faster. On behavior cloning tasks in simulation and physical robots, LeWAM is also competitive with established methods. Interpretability analyses show that LeWAM learns control-relevant representations and action distributions effectively guiding test-time planning and control.

作者：Minghao Fu, Tavis Siebert, Eryk Halicki, Randall Balestriero

来源

本页事实对应的公开页面，以原文为准。

1. arXiv 2610.12818：LeWAM: End-to-End World and Action Modeling with JEPAs访问 2026-10-21