论文/arXiv 2610.06955·25 天前

ROMA：面向真实世界以物体为中心多感官主动感知的 LLM 系统

ROMA用LLM构建多感官推理-交互-反馈闭环，并在真实世界数据与ROMA Bench上验证主动感知能力。

AI 整理

• 数据与采集
• 具身模型
• 评测与安全

原文链接：ROMA: LLM System for Real-World Object-Centric Multi-Sensory Active Perception（arXiv 2610.06955 · 发布日期：2026-10-03）

ROMA: LLM System for Real-World Object-Centric Multi-Sensory Active Perception

ROMA用LLM构建多感官推理-交互-反馈闭环，并在真实世界数据与ROMA Bench上验证主动感知能力。

核心要点

• 现有多感官机器人系统多是融合感知输入，缺少通过交互主动获取缺失证据的能力。
• ROMA整合视觉、音频、触觉和力觉，由LLM判断缺失证据、目标物体、交互方式与感知模态。
• 研究构建ROMI-2K真实世界多感官物体交互数据集，覆盖近2,000个物体和6种原子交互。
• 实验显示ROMA能主动获取缺失证据，处理现有方法难以应对的复杂长链多感官感知任务。

验证范围与局限：验证范围为真实世界多感官物体交互数据和ROMA Bench上的单属性、长时程多属性及意图驱动主动感知任务。

英文摘要 (Abstract)

Humans inherently understand the physical world through an active process. When sensory evidence is insufficient to infer physical properties, we naturally interact with the environment by deciding what information is missing, how to acquire it, and when sufficient evidence has been obtained. In stark contrast, existing multi-sensory robot systems mainly integrate sensory inputs rather than actively acquiring missing evidence through interactions. In this work, we introduce ROMA, an LLM-based system for Real-World Object-Centric Multi-Sensory Active Perception. ROMA integrates vision, audio, tactile, and force sensing into a reasoning-interaction-feedback loop. The model identifies missing evidence and determines the target objects, interactions, and modalities, while a physical interface executes the selected interactions and collects the multi-sensory feedback. To support this capability, we construct ROMI-2K, a large-scale real-world multi-sensory object interaction dataset covering nearly 2,000 objects and 6 atomic interactions with synchronized sensory feedback. Building on these data, we develop a two-stage training framework that aligns sensory modalities and equips the LLM to assess evidence sufficiency, select informative interactions, and reason over the multi-sensory feedback. We further characterize active perception as perception chains, where acquired evidence guides subsequent interactions and reasoning, and establish ROMA Bench to evaluate single-attribute, long-horizon multi-attribute, and intent-driven active perception. Experiments show that ROMA can actively acquire missing evidence and solve complex, long-chain multi-sensory perception tasks that existing methods struggle to handle, laying a strong perceptual foundation for active multi-sensory embodied agents.

作者：Ruoxuan Feng, Yutong Chen, Ruihua Song, Huan Yang, Zhongyuan Wang, Guocai Yao, Di Hu

来源

本页事实对应的公开页面，以原文为准。

1. arXiv 2610.06955：ROMA: LLM System for Real-World Object-Centric Multi-Sensory Active Perception访问 2026-10-21