外观
论文arXiv 2610.2142913 天前
MuSTMM：跨多场景学习声音触发的移动操作
原文链接：MuSTMM: Learning Sound-Triggered Mobile Manipulation across Multiple Scenes（arXiv 2610.21429 · 发布日期：2026-10-15）
MuSTMM: Learning Sound-Triggered Mobile Manipulation across Multiple Scenes
针对未见场景STMM评估不足，提出MuSTMM并引入显式空间记忆导航策略，在未见场景单/多事件设置中验证。
核心要点
- 现有STMM评估多只改变物体布局，对有限建筑结构外的未见场景探索不足。
- MuSTMM结合空间音频渲染与物理交互，覆盖31个架构和物体布局多样的场景。
- 方法将位姿对齐深度累积为空间记忆，投影为自我中心BEV占据图并与声音上下文融合。
- 该策略较现有方法成功率提升2.60%，消融支持空间记忆有效；代码和数据集接收后发布。
验证范围与局限：验证范围为MuSTMM的31个场景，以及场景不重叠划分下的未见场景单事件和多事件设置。
英文摘要 (Abstract)
Sound-Triggered Mobile Manipulation (STMM) requires robots to respond to sound events by navigating to the associated objects and performing appropriate physical interactions. Existing STMM evaluations primarily vary object layouts within limited architectural structures, leaving challenges to unseen scenes insufficiently explored. We introduce Multi-scene Sound-Triggered Mobile Manipulation (MuSTMM), a dataset that combines spatial audio rendering and physical interaction across 31 scenes with diverse architectures and object layouts. MuSTMM contains 14.36 million precomputed binaural Room Impulse Responses (RIRs) and a scene-disjoint split for evaluating single-event and multi-event settings in unseen scenes. We further augment the Navigation Policy Model with explicit spatial memory that complements acoustic cues with observed geometry. Specifically, we accumulate pose-aligned depth observations in a shared coordinate frame and project this memory into an egocentric Bird's-Eye View (BEV) occupancy map. Fusing map features with sound context produces a goal descriptor that provides both target-location and scene-structure priors, guiding navigation to the target region required for subsequent physical interaction. This policy achieves a 2.60% success rate boost over existing methods, while component ablations further support the effectiveness of spatial memory. MuSTMM and the navigation policy provide a testbed for the new challenge of studying sound-triggered mobile manipulation in unseen scenes. The code and dataset will be released upon acceptance.
作者：Hao Ju, Feng Yan, Zhihao Zhong, Heng Yang, Shaofei Huang, Zhedong Zheng
来源
本页事实对应的公开页面，以原文为准。