外观
论文arXiv 2610.1326219 天前
面向视觉-语言-动作策略的双流记忆
原文链接：Dual-Stream Memory for Vision-Language-Action Policies（arXiv 2610.13262 · 发布日期：2026-10-09）
Dual-Stream Memory for Vision-Language-Action Policies
为记忆依赖操作，提出结合时间压缩与相关帧检索的双流记忆，并在RoboMME和真机任务上验证提升。
核心要点
- 问题在于操作策略需跟踪执行状态，并恢复当前不可见的视觉证据。
- 方法用Gated DeltaNet压缩历史为时间令牌，并用任务条件检索器保留相关帧。
- 完整模型在十六个RoboMME任务上平均成功率达49.22%，优于参考方法44.51%。
- 四个真机任务平均成功率达62.5%，杯子交换失败暴露身份跟踪仍待解决。
验证范围与局限：验证范围限于十六个RoboMME任务和四个真机任务。
英文摘要 (Abstract)
Memory-dependent manipulation requires a policy to track execution state and recover visual evidence that is no longer observable. We propose dual-stream memory for vision-language-action policies, assigning these requirements to complementary representations. A task-agnostic Gated DeltaNet compresses history into one temporal token, supervised by progress prediction and action-conditioned feature reconstruction. A task-conditioned retriever retains frames above a fixed relevance threshold; linear multi-scale fusion encodes each frame into sixteen spatial tokens. We then freeze the temporal compressor and jointly optimize retrieval and policy learning. Across sixteen RoboMME tasks, the complete model achieves 49.22% mean success versus 44.51% for the published FrameSamp+Modul reference, improving fourteen tasks and all four suite means. On four real-robot tasks, mean success reaches 62.5% versus 37.5%, with the largest gains on repeated actions requiring execution-stage discrimination. These results support combining execution context with selective visual access, while failures under cup swaps expose persistent identity tracking as an unresolved requirement.
作者：Yang Wang, Ping Liu, Ruiyu Wang, Xuhua Chen, Lingfeng Zhang, Tao Zhang
来源
本页事实对应的公开页面，以原文为准。