论文HF Daily Papers · arXiv 2610.403394 天前

Wizard模型实现零样本实时角色控制与游戏世界生成

Wizard框架通过骨骼几何结构化表征与自适应掩码机制，实现跨750场景178角色的零样本实时视频生成。其显式状态维护机制解耦视觉生成与游戏逻辑，解决具身仿真中动作一致性与泛化难题。

AI 整理

仿真与训练
具身模型

本文由 AI 根据公开内容整理，可能有误，请以原文为准：Wizard: Manipulate Any World in Real Time via Agentic Control（HF Daily Papers · arXiv 2610.40339）

核心内容

针对现有视频世界模型在未见环境中泛化差、实时交互受限的问题，研究提出Wizard智能体游戏世界模型。该模型构建了包含750个场景与178个角色的多样化数据集，支持对角色运动进行实时精确控制，并能在新型场景、角色及动作上实现零样本泛化，突破了传统方法仅支持简单预设模式的局限。

技术上，Wizard利用骨骼与几何信息作为结构化中间代理条件化视频生成模型，结合自适应掩码机制提升信号对齐度。系统引入显式维护游戏状态的智能体框架，将可控世界表征与视觉生成解耦，确保玩家自由配置虚拟世界时保持视觉质量与机械一致性，为具身智能提供高保真可交互仿真环境。

关键看点

结构化表征：构建含750场景178角色数据集，以骨骼几何信息为中间代理条件化生成
对齐优化：采用自适应掩码机制增强条件信号与视频生成的时空对齐精度
状态解耦：智能体框架显式维护更新游戏状态，分离视觉生成与底层控制逻辑
零样本泛化：支持新场景角色动作的零样本迁移，实现实时精细运动控制

工程与产业观察

Wizard验证了结构化物理先验（骨骼/几何）作为视频生成中间层的有效性，解决了纯像素模型缺乏机械一致性的痛点。其显式状态管理与生成解耦架构，为构建可交互、可控制的具身智能训练仿真器提供了关键技术路径，显著降低合成数据与真实物理的Sim-to-Real鸿沟。

英文摘要 (Abstract)

Recent advances in video world models have enabled the generation of increasingly realistic and interactive virtual environments, opening new possibilities for player-customizable gameplay. Agentic frameworks further advance this direction by decoupling explicit game logic and controllable world representations from visual generation. However, existing approaches support only simple game settings and predefined motion patterns, generalize poorly to unseen environments and characters, and offer limited real-time interactivity. In this paper, we introduce Wizard, an agentic game world model that enables real-time, precise control over character motions while generalizing zero-shot across novel scenes, characters, and actions. To achieve this, we construct a diverse dataset spanning 750 scenes and 178 characters, incorporating skeleton and geometry information as structured representations. We leverage these representations as intermediate proxies to condition a video generation model, together with an adaptive masking mechanism that improves alignment between the conditioning signals and generated video rollouts. Furthermore, we introduce an agentic framework that explicitly maintains and updates game states, allowing players to freely configure their virtual worlds and exercise fine-grained control over character motions while preserving visual quality and gameplay consistency. By combining agentic control with structured proxy-based generation, our framework unlocks new possibilities for player-driven game creation with real-time, generalizable, and mechanically consistent world simulation.

Hugging Face Daily Papers 热度：24 upvotes 作者：Yuzhe Wu, ZIJUN LIN, Yeying Jin