外观
论文arXiv 2610.2114713 天前
All-in-One Memory：面向记忆智能体的过程奖励强化学习
原文链接：All-in-One Memory: Process-Rewarded Reinforcement Learning for Memory Agents（arXiv 2610.21147 · 发布日期：2026-10-15）
All-in-One Memory: Process-Rewarded Reinforcement Learning for Memory Agents
AiO-Mem将检索、写入与停止建模为共享证据状态上的轨迹，用过程奖励强化学习优化记忆策略，并在长上下文记忆基准上提升答题准确率。
核心要点
- 现有记忆方法常分开优化检索与构建，缺少统一奖励评估搜索、写入和停止。
- AiO-Mem将异构历史编译为可溯源多视图事件，作为检索与记忆写入的共享基底。
- 统一记忆策略在共享证据状态上展开完整交互轨迹，过程奖励模型对轨迹评分。
- 过程分数被聚合为动作均衡轨迹奖励，并由PA-DAPO用于优化记忆策略。
验证范围与局限：验证范围为长上下文记忆基准；摘要未说明具体数据集或应用部署范围。
英文摘要 (Abstract)
Large language models increasingly rely on external memory to answer queries grounded in long-horizon histories such as multi-session dialogues and lengthy documents. However, existing memory methods often optimize retrieval and memory construction separately, without a shared reward space for evaluating search, writing, and stopping decisions. To address this, we propose All-in-One Memory (AiO-Mem), which organizes these operations as transitions over a shared evidence state and directly evaluates the semantic quality of each action in a unified reward space. AiO-Mem compiles heterogeneous histories into source-traceable multi-view events, providing a shared substrate for retrieval and memory writing. A unified memory policy actor then rolls out complete interaction trajectories over this shared evidence state. A separate trajectory-conditioned process reward model is trained through evidence-grounded supervised fine-tuning and trajectory-level DPO to assign process scores to the collected trajectories. These scores are aggregated into an action-balanced trajectory reward and used by Process-Aligned DAPO (PA-DAPO) to optimize the memory policy. Across long-context memory benchmarks, AiO-Mem consistently improves answer accuracy over both static memory systems and RL-trained memory agents.
作者：Bingbing Wang, Jingjie Lin, Weiming Qiao, Jing Li, Ruifeng Xu
来源
本页事实对应的公开页面，以原文为准。