# WorldSketch：VLM构建运动学世界模型实现零样本移动操作

> 原文链接：https://robotics.hellobike.com/papers/2026-11-03-2611-00513-e1e80c9b
> 频道：papers | 更新日期：2026-11-03 | 标签：auto, arxiv
> 核心摘要：提出WorldSketch范式，VLM主动查询感知工具构建最小运动学世界模型而非直接生成动作。在RoboCasa、LIBERO-PRO及R2R-CE任务中零样本成功率达91%、97%和76%，Token消耗降低6至10倍。

# WorldSketch：VLM构建运动学世界模型实现零样本移动操作

::: v-pre
> 本文由 AI 根据公开内容整理，可能有误，请以原文为准：[WorldSketch: Sketching World Models for Zero-Shot Mobile Manipulation](https://arxiv.org/abs/2611.00513)（arXiv 2611.00513）

## 核心内容

针对现有VLM机器人系统受限于预定义技能库或几何推理能力不足的问题，研究提出WorldSketch新范式。该方法让VLM作为智能体主动调用感知工具，构建并视觉验证仅包含任务相关对象、交互几何及可能运动的最小运动学世界模型，随后由几何规划器联合优化接触与全身运动。

系统具备自改进机制，通过执行反馈验证结果并修正不确定假设。实验显示，WorldSketch在RoboCasa、LIBERO-PRO和R2R-CE导航任务中零样本成功率分别达91%、97%和76%。相比次优基线，其在RoboCasa上成功率提升58个百分点且Token用量减少10倍；在LIBERO-PRO上超越CaP-X 82个百分点；导航任务匹配逐步VLM代理性能但Token消耗降低6倍。

## 关键看点

- **核心范式**：VLM不直接生成控制指令，而是构建最小运动学世界模型供几何规划器优化
- **自改进机制**：利用执行反馈验证预测结果，动态更新不确定假设并修正世界模型
- **性能指标**：RoboCasa零样本成功率91%，较次优基线提升58个百分点且Token减10倍
- **泛化验证**：LIBERO-PRO成功率97%超CaP-X 82%，R2R-CE导航76%且Token降6倍

## 工程与产业观察

该工作验证了将VLM从端到端动作生成转向结构化世界模型构建的有效性，通过解耦语义理解与几何规划，显著降低了大模型推理负载并提升了零样本泛化能力。其自修正机制为解决具身智能Sim-to-Real差距提供了基于在线反馈的轻量化工程路径，对移动操作算法架构设计具有重要参考价值。

## 英文摘要 (Abstract)

Most agentic robotic systems use vision-language models (VLMs) either to orchestrate predefined skills or to directly generate actions, constraints, and control programs. The former is limited by the available skill library, while the latter requires VLMs to reason about precise geometry and contact. In this paper, we propose WorldSketch, a new paradigm for agentic robotics in which a VLM constructs a world model rather than generating robot actions. The VLM acts as an agent that actively queries perception tools to build and visually validate a minimal kinematic world model, capturing only task-relevant objects, their interaction geometry, and possible motions. A geometric planner then jointly optimizes contact and whole-body motion on the resulting model. Moreover, the world model is self-improving: execution feedback verifies outcomes, updates uncertain hypotheses, and enables the agent to revise the model when its predictions are contradicted. We demonstrate the generality of this paradigm across RoboCasa, LIBERO-PRO, and R2R-CE navigation, achieving zero-shot success rates of 91%, 97%, and 76%, respectively. On RoboCasa, WorldSketch outperforms the next-best zero-shot agent baseline by 58 percentage points while using 10 times fewer VLM input tokens; on LIBERO-PRO, it exceeds CaP-X by 82 percentage points. In navigation, it matches the success rate of step-by-step VLM agents while using 6 times fewer input tokens.

作者：Jiaming Wang
:::

## 原始参考来源 (Sources)

- [arXiv 2611.00513：WorldSketch: Sketching World Models for Zero-Shot Mobile Manipulation](https://arxiv.org/abs/2611.00513)