# R2S-Eval：通过视觉-语言模型进行 Real-to-Sim 校准的机器人评估

> 原文链接：https://robotics.hellobike.com/papers/2026-09-03-2609-03276
> 频道：papers | 更新日期：2026-09-03 | 标签：arxiv, cs.RO
> 核心摘要：R2S-Eval用Real-to-Sim校准生成仿真视频，并由VLM偏好评估，在仿真与真实设置中验证机器人策略评估。

# R2S-Eval：通过视觉-语言模型进行 Real-to-Sim 校准的机器人评估

::: v-pre
> 原文链接：[R2S-Eval: Robot Evaluation with Real-to-Sim Calibration via Vision-Language Models](https://arxiv.org/abs/2609.03276)（arXiv 2609.03276 · 发布日期：2026-09-03）

**R2S-Eval: Robot Evaluation with Real-to-Sim Calibration via Vision-Language Models**

R2S-Eval用Real-to-Sim校准生成仿真视频，并由VLM偏好评估，在仿真与真实设置中验证机器人策略评估。

## 核心要点

- 传统真实评估依赖反复硬件试验和人工重置，且成功率难反映执行质量。
- 流程将真实到仿真校准与VLM偏好评估结合，对视频成对比较并汇总排序。
- 实验表明其结论可靠稳定，与人类偏好一致，并减少重复硬件操作。
- 项目页面已提供；方法还能揭示二元成功标签未捕获的行为质量差异。

**验证范围与局限**：验证范围为仿真和真实世界设置，摘要未说明具体机器人数量或任务类型。

## 英文摘要 (Abstract)

Evaluating robot manipulation policies is becoming increasingly important as generalist models, particularly vision-language-action (VLA) models, are deployed on physical robots. However, conventional real-world evaluation remains labor-intensive, unstable, and insufficiently informative. It requires repeated hardware trials, manual scene resets, and continuous operator monitoring, may produce different policy rankings across repeated evaluations, and primarily relies on success-rate metrics that provide limited information about execution quality. In contrast, humans assess robot performance by observing and comparing complete behaviors rather than relying solely on binary success outcomes. To this end, we propose R2S-Eval, an evaluation pipeline that combines real-to-sim calibration with vision-language model (VLM) preference evaluation. The real-to-sim component efficiently generates rollout videos in a simulator calibrated to the real-world evaluation setting, thereby reducing the need for repeated hardware trials. The VLM evaluator assesses the execution quality of rollout videos and produces pairwise preferences, which are subsequently aggregated into policy rankings. We further introduce a protocol to assess whether the proposed evaluation pipeline yields validated policy conclusions while mitigating the key challenges of conventional real-world evaluation. Experiments in both simulation and real-world settings demonstrate that R2S-Eval produces reliable and stable policy conclusions, achieves agreement with human preferences, substantially reduces repeated hardware-operation effort, and reveals behavior-quality differences that are not captured by binary success labels. In general, R2S-Eval advances robot evaluation from manual success counting toward automated, statistically stable, and quality-aware evaluation of robot behavior. Project page: https://r2s-eval.github.io.

作者：Yidi Wang, Feixiang Ruan, Ruoqu Chen, Jie Yin, Yang Yu, Mengdi Xu, Kaifeng Zhang
:::

## 原始参考来源 (Sources)

- [arXiv 2609.03276：R2S-Eval: Robot Evaluation with Real-to-Sim Calibration via Vision-Language Models](https://arxiv.org/abs/2609.03276)