外观
论文arXiv 2607.237973 个月前
用于注意力的记忆：基于 Vision-Language-Motion Map 的语言条件重感知
原文链接：Memory for Attention: Language-Conditioned Re-Perception with a Vision--Language--Motion Map（arXiv 2607.23797 · 发布日期：2026-07-26）
Memory for Attention: Language-Conditioned Re-Perception with a Vision--Language--Motion Map
论文将有限感知预算下的注意力分配表述为重感知，利用 VLMM 持久地图记忆，在 AI2-THOR 场景、CLIP 先验和26个查询上验证。
核心要点
- 持久行为标注地图对空间导航帮助有限，闭环执行收益从约35%降至约4%。
- 方法将重感知建模为注意力决策，用变化历史或最近一次看到的时间调度感知预算。
- 在保留测试中，地图记忆给出最佳调度；无记忆类别先验在 sqrt-law 下最弱。
- 语言条件任务中，VLMM 较相关性加权 recency 基线高2.9%，较类别先验高9.2%。
验证范围与局限：验证范围为28个 AI2-THOR 场景、保留调度测试、CLIP 可移动性先验和26个查询；摘要未提真机实验。
英文摘要 (Abstract)
A robot carrying a persistent, behavior-annotated map faces two very different planning questions, and its memory answers only one of them well. The spatial-navigation question - how to walk around a room - we address first, and report a negative: building on Vision-Language-Motion Maps (VLMM), a behavior-aware planner cost cuts a planning-time objective by ~35% over 28 AI2-THOR scenes, but under closed-loop execution the benefit nearly vanishes (~4%) and an on-demand vision-language model (VLM) does as well. The resource-allocation question is different: under a limited perception budget, what should the robot attend to right now to keep its own map fresh? Framing re-perception as this attention decision, we show a persistent map's memory - change-history, or even just recency of last sighting - yields the best of the re-perception schedules we test (held-out), while the memoryless category prior is the weakest of them under the sqrt-law schedule -- though under a Whittle index it leads memory until heterogeneity is real. The gain grows with per-instance heterogeneity as a Cauchy-Schwarz bound predicts, tracking Var(sqrt(lambda)), the variance of root-volatility, and reallocates budget toward the important objects the schedule protects; against a real CLIP movability prior it is +21-26%, of which roughly half survives once that prior's saturated scale is calibrated (+7-13%). The map's full combination earns its keep when the task is language-conditioned: told what to keep track of, VLMM grounds the relevant objects (open-vocabulary) and tracks their change (memory), beating a relevance-weighted recency baseline (+2.9% over 26 queries, at full heterogeneity; the ordering reverses when instance rates track category norms) - and a category prior (+9.2%). The map earns its keep not by telling the robot how to walk around a room, but by telling it what to pay attention to.
作者：Dibyendu Ghosh
来源
本页事实对应的公开页面，以原文为准。