# WM^MLLM：基于MLLM构建可编辑持久世界模型，环境恢复指标达91.5

> 原文链接：https://robotics.hellobike.com/papers/2026-11-18-2611-24193-590524de
> 频道：papers | 更新日期：2026-11-18 | 标签：auto, arxiv, hf-daily-papers
> 核心摘要：提出首个基于多模态大语言模型的交互式世界模型WM^MLLM，通过外部全景状态与修订历史实现单图持久可编辑生成。在MBench-A环境恢复均值达91.5，相机往返成功率85.3%，显著优于现有基线，解决视频生成漂移难题。

# WM^MLLM：基于MLLM构建可编辑持久世界模型，环境恢复指标达91.5

::: v-pre
> 本文由 AI 根据公开内容整理，可能有误，请以原文为准：[WM^MLLM: Turning MLLMs into World Models](https://arxiv.org/abs/2611.24193)（HF Daily Papers · arXiv 2611.24193）

## 核心内容

针对视频世界模型重访时内容漂移或消失的痛点，研究提出WM^MLLM，这是首个基于多模态大语言模型（MLLM）构建的交互式世界模型。该方案支持从单张图像出发进行持久化、可编辑的世界生成。在MBench-A基准测试中，其四项环境恢复指标均值高达91.5（基线仅为0.1至74.0），相机往返任务完成率达85.3%（基线最高50.9%），并在CompBench全部15项指标中取得最优值，验证了其在长时序一致性上的突破。

技术上，WM^MLLM将世界状态以带修订历史的全景图形式置于生成器外部，由MLLM将用户指令转化为状态操作并校验候选结果，图像与视频生成器仅负责视图提议与呈现。这种架构确保重访同版本时视图精确复现，局部编辑不影响区域外像素。实验显示，生成后选择写入区域使112次盲测编辑完成率从34.8%提升至83.0%；在258项测试中，46次重访均精确恢复历史视图，97次通过校验的局部编辑均保持背景不变，证实了状态管理的鲁棒性。

## 关键看点

- **环境恢复性能**：MBench-A四项指标均值91.5，远超基线0.1-74.0区间，相机往返成功率85.3%对比基线最高50.9%
- **状态管理机制**：采用外部全景图加修订历史存储世界状态，MLLM执行操作并校验，确保重访视图精确复现及局部编辑背景零漂移
- **编辑完成率优化**：引入生成后区域选择机制，使112次盲测局部编辑完成率从34.8%大幅提升至83.0%
- **综合评测表现**：在CompBench全部15列指标中优于15个基线模型，WBench与MBench-A测试中所有通过校验的编辑均保持非目标区域像素不变

## 工程与产业观察

该工作通过将世界状态显式外置并由MLLM进行符号化操作，有效解决了纯生成式世界模型在具身仿真中缺乏持久性与可控性的核心缺陷。其高精度环境恢复与零漂移编辑能力，为构建可长期交互、支持因果验证的机器人训练场提供了关键技术范式，对提升Sim-to-Real迁移可靠性具有重要参考价值。

## 英文摘要 (Abstract)

Video world models keep the world in the context of a generator, so a revisit is synthesized again and earlier edits can drift or disappear. We present WM^MLLM, the first interactive world model built on multimodal large language models (MLLMs), for persistent, editable world generation from a single image. WM^MLLM keeps the world state outside every generator, as a panorama with a revision history, and lets the MLLMs write it: they turn each request into operations on this state, select where a local edit is written, and check every candidate before it becomes the next state. Image generators only propose views, and video only presents them. Users can walk to new places and return, edit objects through text or a box, place subjects, and change lighting, weather or season. By construction, a revisit at the same revision reproduces the earlier view exactly, a local edit leaves every panorama pixel outside its region unchanged, and each request starts from the result of the previous one. Against six world model baselines on MBench-A, WM^MLLM has the highest mean on all four environment-recovery metrics (Epipolar 91.5 versus 0.1 to 74.0) and completes the most camera round trips (85.3% versus at most 50.9%). It has the best value in all 15 CompBench columns against 15 baselines. On 112 local edits judged blind, selecting the region to write after generation raises completion from 34.8% to 83.0%. On 258 WBench and MBench-A items, all 46 revisits recover their earlier views, and all 97 local edits that pass the check leave panorama pixels outside their region unchanged.

Hugging Face Daily Papers 热度：13 upvotes
作者：Baiang Li, Enze Xie, Jincheng Yu, Jin Wang, Haozhe Liu, Yitong Li, Juanxi Tian, Felix Heide
:::

## 原始参考来源 (Sources)

- [HF Daily Papers · arXiv 2611.24193：WM^MLLM: Turning MLLMs into World Models](https://arxiv.org/abs/2611.24193)