外观
论文arXiv 2509.249482025-09-29
World-Env：世界模型驱动VLA后训练，5条演示实现安全强化学习
本文由 AI 根据公开内容整理，可能有误，请以原文为准：World-Env: Leveraging World Model as a Virtual Environment for VLA Post-Training（arXiv 2509.24948）
核心内容
针对模仿学习训练的VLA模型在数据稀缺场景下性能退化及实机强化学习不可重置、高风险的难题，研究团队提出World-Env后训练框架。该方法通过低成本世界模型构建虚拟仿真环境，替代昂贵的物理交互，使VLA能在安全约束下探索并泛化。实验表明，在复杂机器人操作任务中，仅需每任务5条专家演示即可实现显著性能提升，有效解决了传统方法的数据低效与执行冗余问题。
技术架构包含两大核心组件：一是生成时间一致未来视觉观测的物理一致世界模拟器，保障仿真真实性；二是VLM引导的即时反射器（Instant Reflector），提供连续奖励信号并精准预测动作终止点，解决任务完成检测缺失导致的冗余动作问题。该方案将世界模型作为虚拟环境载体，实现了VLA从模仿学习到强化学习的平滑过渡，相关代码已在 github.com/amap-cvlab/world-env 开源，为资源受限场景下的具身智能后训练提供了可扩展范式。
关键看点
- 极低样本需求：突破VLA数据瓶颈，每任务仅需5条专家演示即可通过后训练获得显著性能增益
- 虚拟环境替代：利用物理一致世界模型生成时序连贯视觉观测，规避工业自动化等高风险场景实机RL成本
- VLM即时反馈：集成VLM引导的Instant Reflector模块，提供连续奖励信号并预测动作终止，消除冗余执行
- 开源工程资产：完整代码已在GitHub开源，支持VLA模型在资源受限设置下的安全探索与分布外泛化
工程与产业观察
World-Env验证了“世界模型即仿真器”在VLA后训练中的工程可行性，通过VLM闭环反馈解决了Sim-to-Real中的奖励稀疏与终止判定难题。对于缺乏大规模真机数据的团队，该方案提供了低成本RL微调路径，对工业装配等容错率低场景的具身算法落地具有直接参考价值。
英文摘要 (Abstract)
Vision-Language-Action (VLA) models trained via imitation learning suffer from significant performance degradation in data-scarce scenarios due to their reliance on large-scale demonstration datasets. Although reinforcement learning (RL)-based post-training has proven effective in addressing data scarcity, its application to VLA models is hindered by the non-resettable nature of real-world environments. This limitation is particularly critical in high-risk domains such as industrial automation, where interactions often induce state changes that are costly or infeasible to revert. Furthermore, existing VLA approaches lack a reliable mechanism for detecting task completion, leading to redundant actions that reduce overall task success rates. To address these challenges, we propose World-Env, an RL-based post-training framework that replaces physical interaction with a low-cost world model-based virtual simulator. World-Env consists of two key components: (1) a physically-consistent world simulator that generates temporally consistent future visual observations, and (2) a vision-language model (VLM)-guided instant reflector that provides continuous reward signals and predicts action termination. This simulated environment enables VLA models to safely explore and generalize beyond their initial imitation learning distribution. Our method achieves notable performance gains with as few as five expert demonstrations per task. Experiments on complex robotic manipulation tasks demonstrate that World-Env effectively overcomes the data inefficiency, safety constraints, and inefficient execution of conventional VLA models that rely on real-world interaction, offering a practical and scalable solution for post-training in resource-constrained settings. Our code is available at https://github.com/amap-cvlab/world-env.
作者：Junjin Xiao, Yandan Yang, Xinyuan Chang, Ronghan Chen, Feng Xiong, Mu Xu, Wei-Shi Zheng, Qing Zhang
来源
本页事实对应的公开页面，以原文为准。