外观
论文arXiv 2610.2024014 天前
PAWA：面向多智能体的全景动作条件世界建模
原文链接：PAWA: Panoramic Action-conditioned World Modeling for Multiple Agents（arXiv 2610.20240 · 发布日期：2026-10-14）
PAWA: Panoramic Action-conditioned World Modeling for Multiple Agents
PAWA面向多人共享环境预测，构建动作对齐的POV/全景视频数据与双塔模型，并在Minecraft数据集上验证。
核心要点
- 多人世界模型需预测多玩家独立动作对共享环境及视野外区域的影响。
- PAWA配对每名玩家POV、世界稳定全景视频和对齐动作，并支持重渲染观察配置。
- 双塔模型联合预测POV与全景视频，用跨注意力在共享世界坐标中对齐同方向token。
- 实验显示POV预测优于现有多人世界模型，并生成高保真全景预测；数据和流水线开源。
验证范围与局限：验证范围为Minecraft中多玩家交互会话的POV与全景视频预测。
英文摘要 (Abstract)
A multiplayer world model must predict how the independent actions of several players change a shared environment, including changes outside a player's current field of view. A point-of-view (POV) video captures such changes only after they enter its field of view, whereas panoramic video records the surroundings in every viewing direction and has served as an intermediate representation of the world state in recent world models. Learning this task therefore requires synchronized POV and panoramic videos of interacting players together with their actions, yet no existing dataset provides this combination in Minecraft, an established testbed in which players act independently in a shared, modifiable 3D world. We present PAWA, a dataset and framework for action-conditioned panoramic multiplayer world modeling. The dataset pairs each player's POV video with world-stabilized panoramic video and aligned actions. Its distributed pipeline decouples CPU gameplay capture from GPU rendering, scales beyond 64 simultaneous players, and re-renders each recorded session under new observation configurations without replaying the gameplay. On this dataset, we develop a dual-tower model that jointly predicts POV and panoramic video and couples the two streams through cross-attention that aligns tokens observing the same direction in a shared world frame. We further introduce a quadratic-phase player encoding that provides many more player codes than existing rotary codes at the same embedding dimension. Experiments show that PAWA outperforms state-of-the-art multiplayer world models in POV prediction quality while also producing high-fidelity panoramic predictions. The dataset and collection pipeline are open source to support further research on multiplayer world models.
作者：Siqi Yang, Yimu Wang, Ziqi Cai, Chengxuan Zhu, Shuchen Weng, Erwin Wu, Zhaofei Yu, Kaipeng Zhang, Boxin Shi
来源
本页事实对应的公开页面，以原文为准。