论文/arXiv 2610.20597·14 天前

语言智能体的行动与预测协同演化

针对长程rollout成本高且中间监督利用不足，CoPE用多步动作目标训练策略骨干，并在五个基准验证。

AI 整理

• 本体 SDK 与控制
• 具身模型
• 端侧部署

原文链接：Co-Evolving Acting and Forecasting for Language Agents（arXiv 2610.20597 · 发布日期：2026-10-14）

Co-Evolving Acting and Forecasting for Language Agents

针对长程rollout成本高且中间监督利用不足，CoPE用多步动作目标训练策略骨干，并在五个基准验证。

核心要点

• 问题在于语言智能体RL依赖终端标量奖励，长程rollout昂贵且中间监督利用不足。
• CoPE在成功的on-policy轨迹上，训练策略骨干生成各历史之后的多步动作序列。
• 在ALFWorld、ScienceWorld等五个基准上，CoPE较GRPO和相关RL基线总体更强。
• ScienceWorld上样本效率最高较ECHO和GRPO提升1.9×、2.0×；代码库已公开。

验证范围与局限：验证范围限于摘要所列五个基准，包括模拟环境和现实应用。

英文摘要 (Abstract)

Training language agents is bottlenecked by the high cost of long-horizon rollouts, yet standard agentic reinforcement learning (RL) learns only from a terminal scalar reward, leaving the supervision in intermediate steps largely unused. Prior work exploits this supervision through value targets that estimate per-step success probabilities or observation targets that fit environment response. Both are hard to construct well: (1) value targets come from the same single scalar signal for each step in the trajectory, and (2) observation targets, such as web pages and terminal logs, are long and noisy and reach only one step ahead. We propose CoPE, which adds a third kind of multi-step action targets. On successful on-policy trajectories, CoPE trains the policy backbone to generate the action sequences that follows each history. Each action is also supervised when forecast from the preceding histories, forming a temporal ensemble that the shared backbone absorbs during training, and this enables acting and forecasting to co-evolve. The policy still acts one step at a time at inference, with no overhead. Across five benchmarks (ALFWorld, ScienceWorld, WebShop, $τ^2$-bench, and AppWorld), CoPE (1) achieves stronger aggregate performance than GRPO and RL baselines with predictive auxiliary targets across simulated environments, while improving overall performance on both realistic applications; (2) delivers up to $1.9\times$ and $2.0\times$ higher sample efficiency than ECHO and GRPO, respectively, on ScienceWorld; and (3) achieves these gains with a single policy backbone, without additional models, separate pretraining, or inference overhead. The codebase is available at https://github.com/ulab-uiuc/CoPE.

作者：Haofei Yu, Yixing Chen, Yining Zhao, Xuyan Ye, Yuxiang Liu, Zhigang Hua, Yan Xie, Shuang Yang, Jiaxuan You

来源

本页事实对应的公开页面，以原文为准。

1. arXiv 2610.20597：Co-Evolving Acting and Forecasting for Language Agents访问 2026-10-21