论文/arXiv 2610.09566·21 天前

用于真实机器人操作且对间歇性视觉丢失鲁棒的 Targeted Modality Dropout

针对多模态模仿学习过度依赖视觉的问题，提出基于注意力选择丢弃主导模态的 TMD，并在双臂真机上验证。

AI 整理

• 数据与采集
• 具身模型
• 端侧部署

原文链接：Targeted Modality Dropout for Real-Robot Manipulation Robust to Intermittent Vision Loss（arXiv 2610.09566 · 发布日期：2026-10-07）

Targeted Modality Dropout for Real-Robot Manipulation Robust to Intermittent Vision Loss

针对多模态模仿学习过度依赖视觉的问题，提出基于注意力选择丢弃主导模态的 TMD，并在双臂真机上验证。

核心要点

• 多模态模仿学习易依赖视觉等主导模态，推理时模态丢失会破坏策略执行。
• TMD 用注意力估计各模态依赖度，选择性丢弃最主导模态，并加入熵正则。
• 双臂操纵器真机评估显示，视觉丢失时基线成功率大幅下降，TMD可维持执行。
• 随机丢弃模态且无熵正则的常规 dropout，即使无视觉丢失也在许多任务失败。

验证范围与局限：验证范围为双臂操纵器真机评估及视觉丢失情形，具体任务类型摘要未说明。

英文摘要 (Abstract)

Imitation learning policies that integrate multiple sensory modalities are prone to overreliance on a dominant modality, such as vision, during training, which can disrupt policy execution when that modality is lost at inference time. In this paper, we introduce Targeted Modality Dropout (TMD), in which the dependence on each modality is estimated using attention and the most dominant modality is selectively dropped. This is combined with entropy regularization over the dependence distribution. Through real-robot evaluation using a bimanual manipulator, we show that under vision loss the success rate of the baseline policy drops substantially, whereas TMD sustains task execution. In contrast, a conventional dropout that selects the dropped modality at random, without the entropy regularization, fails on many tasks even without vision loss.

作者：Genki SHIKADA, Kazuki Osamura, Masaru Ide, Tetsuya Ogata, Kanata Suzuki

来源

本页事实对应的公开页面，以原文为准。

1. arXiv 2610.09566：Targeted Modality Dropout for Real-Robot Manipulation Robust to Intermittent Vision Loss访问 2026-10-21