# 舒适地卡住：Bench2Drive 闭环协议中的评估陷阱

> 原文链接：https://robotics.hellobike.com/papers/2026-10-15-2610-21741
> 频道：papers | 更新日期：2026-10-15 | 标签：arxiv, cs.RO
> 核心摘要：指出Bench2Drive闭环评估会误判E2E驾驶模型，提出距离式舒适度修正，并在CARLA仿真八个模型及UniAD、VAD上验证。

# 舒适地卡住：Bench2Drive 闭环协议中的评估陷阱

::: v-pre
> 原文链接：[Comfortably Stuck: Evaluation Pitfalls in Bench2Drive's Closed-Loop Protocol](https://arxiv.org/abs/2610.21741)（arXiv 2610.21741 · 发布日期：2026-10-15）

**Comfortably Stuck: Evaluation Pitfalls in Bench2Drive's Closed-Loop Protocol**

指出Bench2Drive闭环评估会误判E2E驾驶模型，提出距离式舒适度修正，并在CARLA仿真八个模型及UniAD、VAD上验证。

## 核心要点

- Bench2Drive虽是CARLA上主流闭环基准，但其评估协议可能误呈现模型得分。
- 发现舒适度指标有实现错误且分段方案奖励静止，提出基于距离的修正指标。
- 原舒适度与八个模型的静止时间相关r=0.97，修正后相关降至r=0.06。
- 还揭示导航信号供应的两类陷阱，并将发布独立修复代码用于轨迹复算。

**验证范围与局限**：验证范围为Bench2Drive/CARLA闭环评估、八个模型，并在UniAD和VAD上演示导航信号影响。

## 英文摘要 (Abstract)

Closed-loop evaluation is widely regarded as a more faithful way to assess end-to-end (E2E) driving models than open-loop evaluation, and Bench2Drive, built on the CARLA simulator, has become the dominant benchmark for it. We show that its evaluation protocol itself contains pitfalls that can misrepresent the models scored on it. Bench2Drive's comfort metric, adapted from nuPlan, has several implementation errors and a segmentation scheme that rewards standing still, so it mostly measures how long a vehicle stays stationary, correlating with stationary time at r = 0.97 across eight models. We propose a corrected, distance-based metric that cuts that correlation to r = 0.06. Under it, the strong driving-score/comfort tradeoff the original metric reports is no longer detectable once models are compared at matched driving speed, which suggests that comfort trades against how fast a model drives rather than against how well it drives. We also uncover two pitfalls in how the harness supplies navigation signals: the default controller can replace a model's own steering with an angle toward the route target point, and the target point supplied at evaluation time can differ substantially from what a model saw in training. We demonstrate both on UniAD and VAD, where their effects differ sharply, and argue that future closed-loop evaluations should check for them. We will release all fixes as standalone code, so corrected comfort scores can be obtained from logged trajectories without re-running closed-loop simulation.

作者：Thomas Schwarzfischer, Emad Aghajanzadeh, Moritz Thoma, Nael Fasfous, Manoj Rohit Vemparala, Pierpaolo Mori, Daniel Mueller-Gritschneder, Ulf Schlichtmann
:::

## 原始参考来源 (Sources)

- [arXiv 2610.21741：Comfortably Stuck: Evaluation Pitfalls in Bench2Drive's Closed-Loop Protocol](https://arxiv.org/abs/2610.21741)