论文/arXiv 2610.22323·13 天前

Mem4Act：为长程机器人操作学习有效潜在记忆

Mem4Act通过固定大小潜在记忆学习，解决长程机器人操作中的记忆依赖控制，并在RoboMME、RMBench和四项真实任务验证。

AI 整理

• 本体 SDK 与控制

原文链接：Mem4Act: Learning Effective Latent Memory for Long-Horizon Robotic Manipulation（arXiv 2610.22323 · 发布日期：2026-10-15）

Mem4Act: Learning Effective Latent Memory for Long-Horizon Robotic Manipulation

Mem4Act通过固定大小潜在记忆学习，解决长程机器人操作中的记忆依赖控制，并在RoboMME、RMBench和四项真实任务验证。

核心要点

• 长程操作需保留历史交互信息，现有历史、视觉或语言表示面临长度增长或信息丢失问题。
• 方法用多粒度时间聚合覆盖长历史，并以问答监督和关键帧DINO特征重建保留信息。
• 采用memory-first两阶段训练，鼓励动作策略实际依赖固定大小的记忆接口。
• 在RoboMME、RMBench和四项真实任务上达最优，成功率为49.88%、61.0%、81.5%。

验证范围与局限：验证范围为RoboMME、RMBench和四项真实世界操作任务。

英文摘要 (Abstract)

Long-horizon robotic manipulation fundamentally requires memory: agents must retain task-relevant information from past observations and interactions and use it to guide future actions. Existing approaches typically preserve raw historical observations, compressed visual tokens, or language summaries, but these representations either expose the action policy to increasingly long histories or risk discarding task-relevant temporal, semantic, and perceptual information. We introduce Mem4Act, a framework for learning effective latent memory as a fixed-size representation explicitly optimized for long-horizon robotic control. Mem4Act addresses three coupled challenges: how to efficiently cover long histories, what information memory should retain, and how to ensure the policy actually relies on memory. To this end, it organizes history via multi-granularity temporal aggregation, preserves complementary semantic and perceptual information through question-answering supervision and key-frame DINO feature reconstruction, and employs memory-first two-stage training to encourage greater use of memory by the action policy. The resulting memory captures both high-level task state and fine-grained perceptual evidence while providing a bounded interface to the action policy. Across RoboMME, RMBench, and four real-world manipulation tasks, Mem4Act achieves state-of-the-art performance on memory-dependent control, with success rates of 49.88%, 61.0%, and 81.5%, respectively. Extensive ablations further demonstrate the importance of temporal aggregation, complementary memory supervision, and memory-first training.

作者：Yun Zhang, Zhiyu Huang, Tianhui Cai, Chen Tang, Wei Wang, Jiaqi W. Ma

来源

本页事实对应的公开页面，以原文为准。

1. arXiv 2610.22323：Mem4Act: Learning Effective Latent Memory for Long-Horizon Robotic Manipulation访问 2026-10-21