# ACCRUE：面向机器人基础模型的持续离策略强化学习

> 原文链接：https://robotics.hellobike.com/papers/2026-10-13-2610-18596
> 频道：papers | 更新日期：2026-10-13 | 标签：arxiv, cs.RO
> 核心摘要：ACCRUE面向RFM持续学习新任务与保持旧行为，携带轻量演员-评论家并在仿真和真机装配中验证。

# ACCRUE：面向机器人基础模型的持续离策略强化学习

::: v-pre
> 原文链接：[ACCRUE: Continual Off-Policy Reinforcement Learning for Robot Foundation Models](https://arxiv.org/abs/2610.18596)（arXiv 2610.18596 · 发布日期：2026-10-13）

**ACCRUE: Continual Off-Policy Reinforcement Learning for Robot Foundation Models**

ACCRUE面向RFM持续学习新任务与保持旧行为，携带轻量演员-评论家并在仿真和真机装配中验证。

## 核心要点

- 现实部署要求RFM高效学习新任务，同时保留已获得行为。
- ACCRUE跨任务继承轻量演员和评论家，复用先前行为与价值知识。
- 评论家仅适应当前任务转移，演员通过少量保留片段蒸馏保持旧行为。
- 仿真和真机装配中加速后续学习，近乎零遗忘，训练时间降31.0%至68.5%。

**验证范围与局限**：验证限于仿真和真实机器人硬件上的接触丰富装配序列。

## 英文摘要 (Abstract)

Reinforcement learning (RL) post-training improves robot foundation models (RFMs) beyond their demonstrations, but real-world deployment demands more: learning new tasks efficiently while retaining previously acquired behaviors. Existing continual RL methods on top of VLAs mitigate forgetting but rely on on-policy updates with limited forward transfer, whereas off-policy methods improve single-task sample efficiency without addressing learner persistence across tasks. We propose ACCRUE, a continual off-policy RL framework that carries lightweight actor and critic networks across tasks, allowing each stage to reuse behavioral and value knowledge acquired through prior RL. The inherited critic adapts only on current-task transitions and guides current-task actor learning, while prior behavior is preserved by distilling the actor toward cached targets on a small number of retained episodes. Fixed visual features keep the inherited learner and its replay compatible with base-policy updates. Across contact-rich assembly sequences in simulation and on real-robot hardware, ACCRUE accelerates subsequent-task learning with near-zero forgetting and significantly reduces real-robot training time by 31.0\% to 68.5\%.

作者：Changyeon Kim, Kimin Lee, Yashraj Narang, Yijie Guo
:::

## 原始参考来源 (Sources)

- [arXiv 2610.18596：ACCRUE: Continual Off-Policy Reinforcement Learning for Robot Foundation Models](https://arxiv.org/abs/2610.18596)