外观
论文HF Daily Papers · arXiv 2610.2682810 天前
WildAWM开源：纯开源模型合成GUI/CLI混合环境训练通用计算机代理
本文由 AI 根据公开内容整理，可能有误，请以原文为准：Agent World Model in the Wild: Computer-Use Agent Environment Synthesis（HF Daily Papers · arXiv 2610.26828）
核心内容
针对通用计算机使用代理（CUA）在真实世界交互学习面临的安全与扩展瓶颈，研究团队提出WildAWM可扩展合成流水线。该方案完全基于开源模型构建，无需依赖闭源前沿模型即可生成高质量环境。项目成功合成了341个长程环境-任务对，部分任务需超过100步交互才能完成，即便是GPT-6-Astra等顶尖模型解决率也仅为60%，验证了合成数据的高难度与有效性。
技术上，WildAWM构建了同时支持图形用户界面、模型上下文协议（MCP）服务及命令行接口的可执行环境，使智能体能灵活组合视觉交互、工具调用与代码执行。为解决任务合成中的可解性与验证难题，引入基于执行的质量门控机制，结合异构Agent Rollouts、交叉评审一致性检查及反事实探针。在此合成世界中训练的两种规模Qwen模型，在Web、Claw等8个分布外基准测试中均取得显著性能增益。
关键看点
- 多模态交互环境：构建联合支持GUI、MCP服务及CLI的可执行环境，赋能Agent灵活组合视觉感知、工具使用与代码执行能力。
- 执行级质量门控：采用异构Agent试错、交叉Judge一致性及反事实探针三重机制，确保合成任务具备可解性、高难度及验证器可靠性。
- 合成数据泛化验证：341个长程任务对需超100步求解，GPT-6-Astra解决率仅60%；Qwen模型经此训练在8个OOD基准显著提升。
- 全开源技术栈：整套环境合成流水线完全由开源模型驱动，证明无需闭源前沿模型即可生成高质量CUA训练数据，代码已在GitHub开源。
工程与产业观察
WildAWM验证了利用开源模型合成高难度、多模态交互环境以训练具身/计算机代理的可行性。其提出的执行级质量门控与GUI/CLI/MCP联合环境架构，为突破真实世界数据采集瓶颈提供了标准化范式。对于算法工程师而言，该开源资产可直接用于构建Agent仿真训练场，降低对昂贵真实交互数据的依赖，加速通用操作策略的Sim-to-Real迁移研究。
英文摘要 (Abstract)
Agent learning is shifting from learning from human data toward learning through interaction with the world. However, scaling such learning directly in real-world computer systems is challenging and can introduce substantial security risks. We therefore propose Agent World Model in the Wild (WildAWM), a scalable pipeline for synthesizing controllable environments and tasks for training general computer-use agents (CUAs). Whereas existing approaches typically specialize in a single agent interaction interface, we construct executable environments that jointly support graphical user interfaces, model context protocol services, and command-line interfaces, allowing agents to flexibly combine visual interaction, tool use, and code execution. WildAWM is entirely powered by open-source models, demonstrating that high-quality computer-use environments can be generated without proprietary frontier models. Beyond environments, we address three recurring challenges in task synthesis: ensuring solvability, maintaining sufficient difficulty, and producing reliable verifiers, by introducing execution-based quality gates that combine heterogeneous agent rollouts, cross-judge consistency checks, and counterfactual probes of judgment robustness. The resulting pipeline synthesizes 341 long-horizon environment-task pairs that can require over 100 agent steps to solve and remain challenging even for frontier models, with GPT-6-Astra solving only 60% of them. We train two Qwen models of different sizes exclusively in these synthetic worlds. Both models achieve substantial gains across 8 out-of-distribution benchmarks spanning web, claw, and computer-use agents, demonstrating that learning in WildAWM's synthesized worlds generalizes broadly to unseen computer-use tasks. The code is publicly available at https://github.com/MSR-Orchard/WildAWM.
Hugging Face Daily Papers 热度：64 upvotes 作者：Zhaoyang Wang, Qianhui WU, Haorui Wang, Simon Yu, Baolin Peng, Canwen Xu, Hao Cheng, Wenlin Yao, Boyi Liu, Yite Wang, Jianfeng Gao, Huaxiu Yao
来源
本页事实对应的公开页面，以原文为准。