外观
论文HF Daily Papers · arXiv 2610.37175昨天
MLE-World世界模型预测ML实验结果，Agent性能提升22.1%
本文由 AI 根据公开内容整理，可能有误，请以原文为准：MLE-World: Predicting Machine Learning Experiment Outcomes with a Learned World Model（HF Daily Papers · arXiv 2610.37175）
核心内容
针对AI研究Agent在机器学习任务中因模型训练评估耗时导致探索效率低下的瓶颈，研究团队推出MLE-World世界模型。该模型基于Qwen3.5-4B微调，能够根据代码和数据描述直接预测程序的终端输出（含错误追踪、训练日志及验证分数）并给出置信度评分。在17个MLE-bench Lite任务的 held-out 程序测试中，其预测匹配真实输出的比例达77.9%，显著优于大参数提示模型的41.1%。
技术层面，MLE-World将候选方案评估成本从完整训练降为单次推理调用，支撑了Sequential revision、Best-of-N、Pairwise judging、Fusion及UCB风格规则等五种测试时缩放（TTS）策略。系统仅在预测置信度低时触发真实执行，有效平衡了探索与利用。实验表明，相比无世界模型的基线Agent，MLE-World引导的TTS策略在减少真实执行次数的同时，实现了最高22.1%的平均观测性能提升，验证了轻量级世界模型在自动化科研流程中的加速价值。
关键看点
- 预测精度：基于Qwen3.5-4B微调，在MLE-bench Lite上对held-out程序终端输出预测匹配率达77.9%，远超41.1%的大模型Prompting基线
- 推理降本：以单次推理调用替代完整ML训练评估，支持Best-of-N、UCB等五种测试时缩放策略，解决Agent探索算力瓶颈
- 性能增益：通过置信度门控真实执行机制，MLE-World引导的Agent在减少实际训练次数前提下，平均观测性能提升达22.1%
工程与产业观察
该工作证明了轻量级世界模型在具身智能与自动化科研中的关键作用。通过将高成本的物理/仿真交互转化为低成本的心智模拟，MLE-World为机器人策略搜索提供了高效范式。这种“预测-验证”闭环不仅适用于ML实验，更可迁移至机器人Sim-to-Real场景，通过预测动作后果减少真机试错，是提升具身Agent样本效率的重要工程参考。
英文摘要 (Abstract)
AI research agents can propose, execute, and improve programs that solve machine learning tasks. However, executing each candidate program requires training and evaluating ML models, limiting how many candidates an agent can explore at each step. We introduce MLE-World, a world model finetuned from Qwen3.5-4B that predicts a program's terminal output (error traces, training logs, and validation scores) from its code and data description without executing it, together with a confidence score estimating the prediction's correctness. Because a prediction costs one inference call rather than a training run, MLE-World makes test time scaling (TTS) affordable for these agents. We build five TTS policies, guided by MLE-World, that compare several candidates per step by their predicted outputs: Sequential revision, Best-of-N, Pairwise judging, Fusion, and a UCB style rule that also adds an exploration bonus from this confidence score. Since predictions can be wrong, we run a candidate for real only when its confidence score is low. On 17 MLE-bench Lite tasks, MLE-World matches the real terminal output on 77.9% of held out programs, versus 41.1% for much larger prompted models. MLE-World guided TTS achieves up to 22.1% higher observed mean performance than the agent without a world model, with less real execution. Project webpage: https://mle-world.github.io/.
Hugging Face Daily Papers 热度：2 upvotes 作者：Brian Ko, Han Wang, Hejia Geng, Li Erran Li, Sherry Yang
来源
本页事实对应的公开页面，以原文为准。