论文HF Daily Papers · arXiv 2610.40614昨天

L3G-World：潜空间3D高斯记忆实现9FPS实时世界生成

提出L3G-World框架，利用潜空间3D高斯记忆解决长时序探索中的空间一致性与实时性瓶颈。相比最优基线，3D图元减少38.6倍，渲染提速359倍，支持9 FPS实时生成，为具身智能仿真与导航提供高效视觉底座。

AI 整理

仿真与训练
具身模型

本文由 AI 根据公开内容整理，可能有误，请以原文为准：L3G-World: Real-Time Infinite World Generation with Latent 3D Gaussian Memory（HF Daily Papers · arXiv 2610.40614）

核心内容

针对交互式世界模型在长时序探索中难以兼顾空间一致性与实时生成的问题，研究团队提出L3G-World框架。该方法引入Latent 3D Gaussian Memory（L3G），通过记忆更新器与视频扩散Transformer（DiT）交替自回归生成，使记忆容量随场景范围而非视频流长度增长。实验表明，该系统达到SOTA生成质量与长程空间一致性，实现9 FPS实时世界生成，显著突破现有显式3D记忆方法的性能瓶颈。

技术上，L3G结合前馈重建效率与紧凑可渲染场景表示，每个高斯存储VAE潜特征，支持直接渲染至DiT潜空间而无需RGB重编码，规避了传统方法因重编码导致的延迟。量化对比显示，L3G-World相比最高效的3D记忆基线，3D图元数量减少38.6倍，记忆更新速度提升3.2倍，记忆渲染速度提升359倍。这种轻量化表征与潜空间直连机制，使其成为具身智能体实时环境建模与策略验证的高效引擎。

关键看点

架构创新：采用记忆更新器与DiT交替自回归机制，记忆规模仅随场景物理范围扩展，解耦生成时长与显存占用
潜空间直渲：高斯节点内嵌VAE潜特征，直接注入DiT潜空间，消除RGB重编码开销，渲染速度较基线提升359倍
极致轻量化：3D图元数量较最优基线减少38.6倍，记忆更新提速3.2倍，支撑9 FPS实时交互式世界生成
长程一致性：在保持SOTA生成质量前提下，解决长时序探索中的空间漂移问题，适配机器人导航与仿真需求

工程与产业观察

L3G-World通过潜空间3D高斯记忆重构世界模型范式，将渲染提速359倍并实现9 FPS实时生成，解决了具身仿真中长时序空间一致性与计算效率的矛盾。该方案为机器人Sim-to-Real训练、在线SLAM及端到端导航提供了低延迟、高保真的视觉生成底座，大幅降低合成数据生产成本。

英文摘要 (Abstract)

Interactive world models generate visual content conditioned on user actions, yet maintaining spatial consistency over long-horizon exploration in real time remains challenging. Existing approaches rely on either temporal context or explicit 3D memory, both of which scale poorly because memory grows with the length of the generated stream. Moreover, explicit 3D memories condition latent video generators by re-encoding RGB renderings, which precludes real-time generation. In this paper, we introduce a Latent 3D Gaussian Memory (L3G) that combines the efficiency of feed-forward reconstruction with a compact, rapidly renderable scene representation. Building on this memory, we propose L3G-World, in which a memory updater and a video diffusion transformer (DiT) alternate autoregressively to generate spatially consistent worlds. The memory updater spawns Gaussians in newly observed regions and refines existing ones, so that the memory grows with scene extent rather than stream length. Each Gaussian further stores a VAE latent feature, allowing the memory to be rendered directly into the DiT's latent space without VAE re-encoding. Experiments show that L3G-World achieves state-of-the-art generation quality and long-horizon spatial consistency, with up to 38.6times fewer 3D primitives, 3.2times faster memory updates, and 359times faster memory rendering relative to the most efficient prior 3D-memory baseline for each metric, enabling real-time world generation at 9 FPS.

Hugging Face Daily Papers 热度：29 upvotes 作者：Jinhyeok Choi, JoungBin Lee, Jongmin Lee, Minkyeong Jeon, Jiyoung Kim, Kaeun Han, Yoonjin Yoon, Seungryong Kim