外观
论文HF Daily Papers · arXiv 2610.339663 天前
InteractiveWorld：首个面向具身AI助手的可验证人机闭环仿真与评测基准
本文由 AI 根据公开内容整理，可能有误，请以原文为准：InteractiveWorld: A Verifiable Human-Centric Environment for Building Better Interactive MLLM Assistants（HF Daily Papers · arXiv 2610.33966）
核心内容
针对多模态大模型（MLLM）作为第一人称AI助手缺乏可验证测试环境的问题，研究团队发布InteractiveWorld，这是首个支持AI助手与人类Agent进行闭环交互的可验证仿真平台。配套推出的InteractiveBench包含6大类120项多轮交互任务，通过状态奖励机制量化评估模型在真实物理场景中的辅助能力，填补了以人为中心而非机器人中心的评测空白。
该系统架构区别于传统机器人仿真器，采用“AI指令-人类执行-环境反馈”的动态循环机制，模拟穿戴设备视角下的实时协作。基于此基准对12个前沿MLLM的实测表明，即便是表现最优的Claude Fable 5.1，在长程引导和规划任务上的成功率也分别仅为20%和25%。这一量化结果证实现有模型尚不具备可靠的物理世界交互辅助能力，为具身智能算法迭代提供了严谨的验证底座。
关键看点
- 闭环仿真架构：构建AI指导人类Agent、环境实时更新状态的可验证交互闭环，突破传统机器人中心仿真局限
- 动态评测基准：InteractiveBench涵盖6类120项多轮任务，采用基于状态的可验证奖励机制评分
- 前沿模型瓶颈：12个主流MLLM实测显示，最强模型在长程引导与规划任务成功率上限仅为20%和25%
- 第一人称视角：专注于穿戴式设备egocentric场景，验证AI在物理世界中实时辅助人类的实际效能
工程与产业观察
该工作将具身智能评测从“机器人自主操作”拓展至“人机协作辅助”维度，揭示了MLLM在理解物理常识与长程任务规划上的巨大差距。对于研发AR/VR助手及人形机器人交互算法的团队，InteractiveBench提供了标准化的Sim-to-Real验证工具，提示行业需从单纯追求模型参数规模转向强化物理世界交互数据的闭环训练与状态对齐。
英文摘要 (Abstract)
People have long dreamed of an assistant that is always there to guide them and remind what they forget, from new dishes to lost keys. Powered by multimodal large language models (MLLMs), egocentric AI assistants on wearable devices are bringing this dream within reach: they see what the user sees and give instructions for user's physical work when necessary. However, testing such assistance is challenging and usually requires real people. The expensive and rare real interaction data leaves a key question unanswered: Can today's MLLMs truly help people in physical world? To answer this question, we propose InteractiveWorld, the first verifiable environment for closed-loop interaction between human and AI assistant. Unlike existing robot-centric simulators, InteractiveWorld lets the assistant guide a human agent who observes the scene, understands instructions, and acts independently. It thus realistically simulates how AI assistant collaborates with human: the assistant instructs, the human acts, and the environment executes actions and updates what they see next. Building on it, we introduce InteractiveBench, the first dynamic benchmark for egocentric AI assistants, with 120 multi-turn tasks across six categories. In each task, the assistant guides the human in real time, and every instruction shapes the outcome, which is scored by verifiable, state-based rewards. Experiments with 12 frontier MLLMs show they are still far from reliable assistants. Even the strongest model, Claude Fable 5.1, achieves at most 20% and 25% on long-horizon Guidance and Planning tasks, respectively. Overall, our work offers a verifiable platform and a rigorous evaluation set, laying a solid foundation for research on interactive MLLM assistants that truly help people in the physical world.
Hugging Face Daily Papers 热度：92 upvotes 作者：Qinsi Wang, Wei Wen, Chong Zhou, Saksham Suri, Zechun Liu, Lemeng Wu, Hai Helen Li, Yiran Chen, Raghuraman Krishnamoorthi, Qi Qian
来源
本页事实对应的公开页面，以原文为准。