# All-in-One Memory：面向记忆智能体的过程奖励强化学习

> 原文链接：https://robotics.hellobike.com/papers/2026-10-15-2610-21147
> 频道：papers | 更新日期：2026-10-15 | 标签：arxiv, cs.RO
> 核心摘要：AiO-Mem将检索、写入与停止建模为共享证据状态上的轨迹，用过程奖励强化学习优化记忆策略，并在长上下文记忆基准上提升答题准确率。

# All-in-One Memory：面向记忆智能体的过程奖励强化学习

::: v-pre
> 原文链接：[All-in-One Memory: Process-Rewarded Reinforcement Learning for Memory Agents](https://arxiv.org/abs/2610.21147)（arXiv 2610.21147 · 发布日期：2026-10-15）

**All-in-One Memory: Process-Rewarded Reinforcement Learning for Memory Agents**

AiO-Mem将检索、写入与停止建模为共享证据状态上的轨迹，用过程奖励强化学习优化记忆策略，并在长上下文记忆基准上提升答题准确率。

## 核心要点

- 现有记忆方法常分开优化检索与构建，缺少统一奖励评估搜索、写入和停止。
- AiO-Mem将异构历史编译为可溯源多视图事件，作为检索与记忆写入的共享基底。
- 统一记忆策略在共享证据状态上展开完整交互轨迹，过程奖励模型对轨迹评分。
- 过程分数被聚合为动作均衡轨迹奖励，并由PA-DAPO用于优化记忆策略。

**验证范围与局限**：验证范围为长上下文记忆基准；摘要未说明具体数据集或应用部署范围。

## 英文摘要 (Abstract)

Large language models increasingly rely on external memory to answer queries grounded in long-horizon histories such as multi-session dialogues and lengthy documents. However, existing memory methods often optimize retrieval and memory construction separately, without a shared reward space for evaluating search, writing, and stopping decisions. To address this, we propose All-in-One Memory (AiO-Mem), which organizes these operations as transitions over a shared evidence state and directly evaluates the semantic quality of each action in a unified reward space. AiO-Mem compiles heterogeneous histories into source-traceable multi-view events, providing a shared substrate for retrieval and memory writing. A unified memory policy actor then rolls out complete interaction trajectories over this shared evidence state. A separate trajectory-conditioned process reward model is trained through evidence-grounded supervised fine-tuning and trajectory-level DPO to assign process scores to the collected trajectories. These scores are aggregated into an action-balanced trajectory reward and used by Process-Aligned DAPO (PA-DAPO) to optimize the memory policy. Across long-context memory benchmarks, AiO-Mem consistently improves answer accuracy over both static memory systems and RL-trained memory agents.

作者：Bingbing Wang, Jingjie Lin, Weiming Qiao, Jing Li, Ruifeng Xu
:::

## 原始参考来源 (Sources)

- [arXiv 2610.21147：All-in-One Memory: Process-Rewarded Reinforcement Learning for Memory Agents](https://arxiv.org/abs/2610.21147)