Skip to content

论文arXiv 2611.175295 天前

残差适配器优化冻结视觉世界模型规划,DotWall成功率升至98% ​

本文由 AI 根据公开内容整理,可能有误,请以原文为准:Learning Latent Goal Costs for Visual World-Model Planning with an Expanding Curriculum(arXiv 2611.17529)

核心内容 ​

研究针对图像目标导航中冻结视觉世界模型依赖预训练编码器嵌入均方误差(MSE)作为代价函数时,易因视觉相似性误判障碍物后状态的问题,提出在冻结I-JEPA嵌入层上拟合小型残差适配器的解决方案。该方法利用规划器从目标附近逐步扩展初始状态区域收集轨迹,仅依据成功轨迹观测顺序及失败回滚推断进行比较学习,无需任务特定进度标签。在DotWall和PointMaze基准测试中,50组同配置对比实验显示成功率分别从54%提升至98%、82%提升至96%。

技术方案核心在于利用规划经验修正潜在目标代价,而非重训视觉编码器或世界模型。适配器通过学习轨迹时序逻辑,使潜在代价排序与最短合法路径排序的一致性在16个未见DotWall布局中从71.9%提升至85.8%。该机制证明了在不破坏预训练表征的前提下,通过轻量级适配与课程学习策略,可有效解决视觉相似性导致的规划陷阱,为具身智能体在复杂环境下的零样本或少样本目标到达提供了低算力成本的优化范式。

关键看点 ​

  • 残差适配机制:在冻结I-JEPA嵌入层叠加小型残差适配器,利用轨迹时序排序信号优化潜在目标代价,避免视觉相似性误导规划
  • 无标签课程学习:仅依赖成功轨迹观测顺序与失败回滚推断进行训练,无需任务特定进度标签,具备跨任务泛化能力
  • 规划性能跃升:在相同世界模型与规划器设置下,DotWall任务成功率从54%增至98%,PointMaze从82%增至96%
  • 路径一致性验证:在16个未见布局测试中,潜在代价排序与最短合法路径排序一致率从71.9%提升至85.8%

工程与产业观察 ​

该工作验证了在冻结大模型表征上通过轻量级适配与规划经验反馈修正代价函数的有效性,为具身智能体提供了低成本优化视觉导航的路径。对于算力受限或需保持预训练模型稳定性的端侧部署场景,这种“不动底座、只修接口”的策略比全量微调更具工程落地价值,尤其适用于解决视觉欺骗性环境中的长程规划难题。

英文摘要 (Abstract) ​

Image-goal planning with a frozen visual world model depends on how predicted observations are scored against the goal. Mean-squared error (MSE) between embeddings from a pretrained visual encoder can provide a useful signal near a goal, yet favor states across an obstacle that are visually similar to the goal and difficult to reach. We fit a small residual adapter on top of frozen I-JEPA embeddings to improve this latent goal cost. Starting near each goal, a planner collects trajectories from progressively wider regions of initial states. The adapter learns from the order of observations along successful trajectories, with additional comparisons inferred from failed rollouts. No task-specific progress labels are used for fitting, so the procedure can be applied to other goal-reaching tasks. DotWall and PointMaze are used in this work. In paired 50-case evaluations with identical starts, goals, world models, and planner settings, success increases from 54% to 98% in DotWall and from 82% to 96% in PointMaze. On 16 unseen DotWall layout-goal cases, the adapter fitted on one fixed layout improves agreement between latent-cost ordering and shortest legal-path ordering from 71.9% to 85.8%. The results indicate that planning experience can improve a frozen visual representation's goal cost without retraining the encoder or the world model.

作者:Konstantinos Mylonas, Thrasyvoulos Spyropoulos

基于本页技术方向关联的整机参数、核心零部件、实测报告与同赛道选型横评。

外部来源

本页事实对应的公开页面,以原文为准。(点击将离开本站)

  1. arXiv 2611.17529:Learning Latent Goal Costs for Visual World-Model Planning with an Expanding Curriculum ↗访问 2026-11-16