论文/arXiv 2610.04301·25 天前

EnvDreamer：面向具身 AI 的大规模多模态到环境生成

提出EnvDreamer，利用语言与视觉语言模型生成UE5环境，并在导航、重排和操作基准上验证策略训练效果。

AI 整理

• 感知与导航
• 数据与采集
• 具身模型

原文链接：EnvDreamer: Large-Scale Multimodal-to-Environment Generation for Embodied AI（arXiv 2610.04301 · 发布日期：2026-10-03）

EnvDreamer: Large-Scale Multimodal-to-Environment Generation for Embodied AI

提出EnvDreamer，利用语言与视觉语言模型生成UE5环境，并在导航、重排和操作基准上验证策略训练效果。

核心要点

• 动机是将大数据与高容量模型带来的进展扩展到具身学习、世界模型和机器人。
• 方法利用大型语言模型和视觉语言模型，生成可交互、可定制且通过验证的UE5虚拟环境。
• 在生成环境中训练的策略无需显式建图或人工任务监督，在多个具身基准上取得有竞争力结果。
• 发布EnvDreamer-20k，包含2万个通过验证的环境及任务程序、场景图、轨迹和元数据。

验证范围与局限：验证范围为生成的虚拟环境和多个具身基准，摘要未提及真机实验。

英文摘要 (Abstract)

Large datasets and high capacity models have accelerated progress in vision and language. This work introduces a platform aimed at bringing comparable gains to embodied learning, world models, and robotics. We present EnvDreamer, a framework that uses large language and vision language models to generate Unreal Engine 5 environments for embodied AI and robot training. EnvDreamer enables sampling of large, diverse, interactive, customizable, and validator passed virtual environments for training and evaluation across navigation, interaction, and manipulation. We illustrate the platform with a large set of generated scenes and simple baselines. Policies trained on EnvDreamer generated environments, without explicit mapping or human task supervision, achieve competitive results on multiple embodied benchmarks spanning navigation, rearrangement, and manipulation. EnvDreamer also supports image-conditioned reconstruction for real-to-sim studies. Finally, we release EnvDreamer-20k, a dataset of 20,000 validator passed environments with task programs, scene graphs, trajectories, and metadata to support reproducible benchmarking.

作者：Kabir Swain, Sijie Han, Antonio B. Torralba

来源

本页事实对应的公开页面，以原文为准。

1. arXiv 2610.04301：EnvDreamer: Large-Scale Multimodal-to-Environment Generation for Embodied AI访问 2026-10-21