# 超越成功率：视觉-语言-动作系统中记忆的干预式评估

> 原文链接：https://robotics.hellobike.com/papers/2026-10-16-2610-23372
> 频道：papers | 更新日期：2026-10-16 | 标签：arxiv, cs.RO
> 核心摘要：提出 IMEP 在 VLA 系统中干预评估记忆作用，并在多类记忆与 ChainBench 上验证其影响。

# 超越成功率：视觉-语言-动作系统中记忆的干预式评估

::: v-pre
> 原文链接：[Beyond Success Rate: Interventional Evaluation of Memory in Vision-Language-Action Systems](https://arxiv.org/abs/2610.23372)（arXiv 2610.23372 · 发布日期：2026-10-16）

**Beyond Success Rate: Interventional Evaluation of Memory in Vision-Language-Action Systems**

提出 IMEP 在 VLA 系统中干预评估记忆作用，并在多类记忆与 ChainBench 上验证其影响。

## 核心要点

- 仅做成功率消融会同时改变信息和计算，难以证明记忆中的任务相关事实被使用。
- IMEP 在检索接口比较目标事实替换、事实保持替换、微小数值扰动和精确副本。
- 跨回合 learned-bank 实验中，ULP 干预使 11.9% 的种子配对结果发生变化。
- ChainBench 要求使用当前观测缺失的历史信息，完整记忆将成功率从 39.1% 提升到 74.3%。

**验证范围与局限**：验证涵盖回合内、跨回合与 agentic memory，摘要未说明仿真或真机平台。

## 英文摘要 (Abstract)

When we say that a robot uses memory, does that memory actually play a role? Success-rate ablations alone cannot establish the use of task-relevant facts: they change information and computation together, while aggregate rates can hide offsetting gains and losses. We propose an Interventional Memory Evaluation Protocol (IMEP) for vision-language-action (VLA) systems to address four questions: (1) Does memory provide a net benefit? (2) Under what conditions does memory help? (3) Do changes to memory alter task outcomes? (4) Are these effects specific to task-relevant facts? At the retrieval interface, we compare target-fact substitutions with fact-preserving replacements, minimal numerical perturbations, and exact replicas, while assessing decision correctness separately from motor execution success. We evaluate within-episode, cross-episode, and agentic memory. In cross-episode learned-bank experiments, a one-unit-in-the-last-place (ULP) intervention, including its installation procedure, changes 11.9% of seed-paired outcomes, while changing key facts produces no clear additional effect over fact-preserving replacements. To examine when memory helps, we introduce ChainBench, where correct decisions require historical information unavailable in current observations. The full memory configuration raises task success from 39.1% to 74.3%, matching the oracle-decision condition under shared recorded executions. These results show that correct history can support robot decisions, but establishing whether a memory system realizes this benefit requires fact-specific evidence beyond aggregate success rates.

作者：罗科元, Yi Xie, Wanyun Ling, Yuanzhe Li, Ziyue Li, Zhuoqi Zeng
:::

## 原始参考来源 (Sources)

- [arXiv 2610.23372：Beyond Success Rate: Interventional Evaluation of Memory in Vision-Language-Action Systems](https://arxiv.org/abs/2610.23372)