论文/arXiv 2610.15228·18 天前

从词语到世界：在交互式物理环境中基准评测 LLMs

提出闭环物理交互基准，让 LLMs 控制连接真实组件的系统，并在代表性 LLMs 实验中评估其物理行动能力。

AI 整理

• 本体 SDK 与控制
• 具身模型
• 评测与安全

原文链接：Words to World: Benchmarking LLMs in Interactive Physical Environments（arXiv 2610.15228 · 发布日期：2026-10-10）

Words to World: Benchmarking LLMs in Interactive Physical Environments

提出闭环物理交互基准，让 LLMs 控制连接真实组件的系统，并在代表性 LLMs 实验中评估其物理行动能力。

核心要点

• 现有基准多依赖静态文本、离线工具调用或仿真任务，难以衡量真实物理环境中的行动能力。
• 该基准让模型控制连接机械臂、相机和传感器的系统，根据物理反馈持续调整动作。
• 评测覆盖扎根感知、动作规划、工具选择、空间推理、反馈纠错与执行不确定性鲁棒性。
• 实验刻画代表性 LLMs 将语言推理转化为物理动作的能力，并识别多类常见失败模式。

验证范围与局限：验证范围为连接机械臂、相机和传感器等真实组件的闭环物理交互任务。

英文摘要 (Abstract)

Words are increasingly becoming an interface for action: large language models (LLMs) can now invoke tools, operate devices, and make decisions that affect the physical world. However, current benchmarks largely evaluate LLMs through static text responses, offline tool calls, or simulated tasks, leaving open the question of how well they can act in real physical environments. We present a benchmark for measuring LLMs in closed-loop physical interaction, where models control a harness system connected to real-world components including robotic arms, cameras, and sensors. In each task, the model must issue instructions, receive physical feedback from the environment, interpret the resulting state, and adapt its next action accordingly. This setting evaluates capabilities beyond language understanding, including grounded perception, action planning, tool selection, spatial reasoning, feedback-driven correction, and robustness to execution uncertainty. Through experiments with representative LLMs, we characterize their ability to translate linguistic reasoning into physical actions and identify common failure modes such as hallucinated state changes, poor recovery from unexpected outcomes, and unstable long-horizon plans. Our benchmark aims to provide a systematic foundation for evaluating LLMs as interactive agents that connect words, tools, and the physical world.

作者：Xinnan Dai, Kaiqi Yang, Yaxin Li, Cheng Luo, Wanli Li, Mingqing Xiao, Xinyang Jiang, Daoan Zhang, Kaitao Song, Fengze Liu

来源

本页事实对应的公开页面，以原文为准。

1. arXiv 2610.15228：Words to World: Benchmarking LLMs in Interactive Physical Environments访问 2026-10-21