论文HF Daily Papers · arXiv 2610.0085412 天前

Embodied Agent Arena评测：闭源VLM领跑具身任务，开源模型规划操控短板显著

研究发布Embodied Agent Arena基准，涵盖1000案例评测10款VLM。结果显示闭源模型在几何、规划及操控五项指标全面领先；开源模型虽具备空间推理能力，但在任务规划与物理操控成功率上差距明显，且增加计算量无法均匀修复执行失败。

AI 整理

具身模型
评测与安全

本文由 AI 根据公开内容整理，可能有误，请以原文为准：Are Frontier VLM Agents Ready to Be Robot Generalists? An Empirical Study with the Embodied Agent Arena（HF Daily Papers · arXiv 2610.00854）

核心内容

针对前沿视觉语言模型（VLM）是否具备机器人通才能力的争议，研究团队推出Embodied Agent Arena评测基准。该基准整合32个现有数据源并新增168例GeoProbe测试集，共计1000个案例，覆盖几何感知、空间推理、功能可供性、任务规划及物理操控五大核心维度。评测采用最小化测试框架，通过原生接口直接评估10款VLM智能体的端到端任务完成能力，揭示了当前模型在真实具身场景下的性能边界。

评测表明闭源模型在所有领域均优于开源模型，后者虽有一定空间推理能力但规划与操控成功率低。研究发现相似的几何误差背后隐藏着不同的尺度与形状畸变，且有效区域估计不等于可用动作点识别。执行阶段常出现子目标达成后失去物体控制或动作绑定错误等问题。值得注意的是，单纯增加计算量不能解决所有故障：代码修复可能恢复执行但未完成任务，自我反思甚至可能推翻正确判断，且强规划模型的失败成本集中在长时序身体控制任务中。

关键看点

评测基准构建：Embodied Agent Arena包含1000案例，新增168例GeoProbe数据集专门测试相机运动、深度及尺度估计，采用最小化harness通过原生接口评估VLM。
开闭源性能差异：闭源模型在几何、空间推理、可供性、规划及操控五项指标全面领先；开源VLM仅展现基础空间推理能力，任务规划与物理操控成功率显著偏低。
执行失效模式：智能体常在达成早期子目标后丢失物体控制、将动作绑定至错误接收区域或在物理状态未满足时提前终止，且有效区域估计并不等同于可执行动作点识别。
计算扩展局限性：增加计算量无法均匀修复失败，代码修复可恢复执行但未必完成任务，自我反思机制可能逆转正确判断；强规划模型的失败Token消耗集中于长时序身体控制任务。

工程与产业观察

该评测揭示了VLM从语义理解到物理执行的鸿沟：开源模型在空间推理上的进步并未转化为可靠的操控与规划能力。工程落地需警惕“计算缩放”陷阱，单纯增加推理算力或自我反思可能引入新噪声。建议开发者关注GeoProbe等细粒度感知基准，将优化重心从纯文本规划转向物理状态验证与闭环纠错机制，而非盲目追求大参数模型。

英文摘要 (Abstract)

A robot generalist must interpret a scene, act on it, and satisfy every condition of a task. How far do frontier vision-language models (VLMs) meet these demands? We introduce Embodied Agent Arena, evaluating ten VLM agents across Geometry, Spatial Reasoning, Affordance, Task Planning, and Manipulation. Its 1,000 cases combine 32 established sources with GeoProbe, our 168-case benchmark for camera, motion, depth, and scale estimation. A minimal harness leaves task decisions to the model through source-native interfaces. Closed-weight models lead all five domains, whereas open-weight VLMs show useful spatial reasoning but substantially lower planning and manipulation success. Beneath these averages, similar geometric errors conceal different scale and shape distortions, and valid region estimates need not identify usable action points. During execution, agents can achieve early subgoals yet lose object control, bind an action to the wrong receiving region, or stop before the required physical state holds. Additional computation does not uniformly resolve these failures: code repair can restore execution without completing the task, while self-review can reverse correct judgments. Failure costs also differ: the strongest planners concentrate failed-token expenditure in prolonged body-control tasks, whereas other models fail broadly across procedures. Matched traces show how successful recovery uses feedback to restore a missing prerequisite and advance the remaining task.

Hugging Face Daily Papers 热度：3 upvotes 作者：Haojian Huang, Pukun Zhao, Zexi Li, Yehang Zhang, Yangkai Wei, Wenqian Li, Han Yang, Kaiwen Zhou, Ying-Cong Chen, Yinchuan Li