论文HF Daily Papers · arXiv 2611.248694 天前
WorldRuler:基于3D规则图实现可配置交互式世界模型生成
本文由 AI 根据公开内容整理,可能有误,请以原文为准:WorldRuler: Interactive World Modeling under Configurable Rules(HF Daily Papers · arXiv 2611.24869)
核心内容
针对生成式世界模型难以遵循用户自定义规则的挑战,论文提出 WorldRuler 框架,解决了规则状态持久化(Rule-State Persistence)与规则到视觉实现(Rule-to-Visual Realization)两大核心难题。该研究构建了包含详细状态与事件标注的 RuleVerse 数据集,涵盖 GTA-V 和 Habitat 环境,特别包含规则在受影响对象可见前即生效的边缘案例,为具身智能交互仿真提供了标准化的评测基准与数据支撑。
技术上,WorldRuler 采用可执行 3D 规则图组织规则定义、依赖关系及对象状态,支持规则与世界状态的联合保存与恢复。系统通过动态对象绑定将规则实体映射至视觉实例,并利用存储状态约束生成过程;引入对象属性状态替换与反事实重建训练策略,确保在改变指定对象属性与运动时保持外观兼容,同时限制对无关内容的干扰,实现了规则驱动下的精准视觉生成闭环。
关键看点
- 3D规则图架构:构建 executable 3D rule graph 统一管理规则定义、依赖及对象状态,支持跨会话的规则与世界状态联合保存与精确恢复
- 动态视觉绑定:通过动态对象绑定机制将规则实体接地至视觉实例,利用历史状态约束生成模型,解决不可见对象受规则影响后的视觉一致性问题
- 反事实重建训练:引入对象属性状态替换与反事实重建训练策略,实现对指定对象属性的精准控制,在保证外观运动兼容的同时抑制对无关内容的生成干扰
- RuleVerse数据集:基于 GTA-V 和 Habitat 构建自动化数据采集管线,提供包含规则先于对象可见等复杂交互场景的详细状态与事件标注数据集
工程与产业观察
该工作突破了传统世界模型仅能模拟物理规律的局限,为具身智能提供了支持自定义逻辑的可配置仿真环境。其 3D 规则图与反事实训练范式,有效解决了长时序交互中的状态漂移与视觉幻觉问题,对构建高保真、可验证的机器人Sim2Real训练场具有重要工程参考价值。
英文摘要 (Abstract)
Rules govern how player actions change interactive worlds. Different rules create distinct gameplay experiences within similar scenes and action spaces. For customizable gameplay, generative world models must follow user-defined rules beyond real-world behaviors. However, realizing user-defined rules presents two challenges: Rule-State Persistence and Rule-to-Visual Realization. The former requires preserving rule definitions, dependencies, and world states across saving and loading, so that restored worlds follow the established rules. The latter requires accurately realizing rule-induced changes on designated objects and attributes, particularly when affected objects are unseen. In this paper, we propose WorldRuler, which organizes rule definitions, dependencies, and object states in an executable 3D rule graph and connects rule execution, video generation, and perceptual feedback in a closed loop. The graph supports joint saving and restoration of rules and world states, while dynamic object binding grounds rule-specified entities in visual instances and uses stored states to constrain generation from their first appearance. We further introduce object attribute state replacement and counterfactual reconstruction training, enabling accurate attribute control and compatible appearance and motion changes while limiting interference with unrelated content. We develop an automated data collection and annotation pipeline using GTA-V and Habitat to construct RuleVerse, a dataset of rule-driven interactions with detailed state and event annotations, including cases where rules take effect before affected objects become visible. Extensive experiments demonstrate the effectiveness of WorldRuler in preserving rule-state consistency across saving and loading and accurately realizing rule outcomes on designated objects and attributes.
Hugging Face Daily Papers 热度:16 upvotes 作者:Mingliang Zhai, Chuanhao Li, Jiaming Tan, Zhen Li, Yuwei Wu, Yunde Jia, Zian Meng, Yuanyang Yin, Kaipeng Zhang
外部来源
本页事实对应的公开页面,以原文为准。(点击将离开本站)