Skip to content

论文arXiv 2611.191385 天前

SuperGlance:VLM隐状态驱动策略规划耗时84ms提速6倍 ​

本文由 AI 根据公开内容整理,可能有误,请以原文为准:SuperGlance: Driving at a Glance with a Vision-Language Model(arXiv 2611.19138)

核心内容 ​

针对视觉-语言-动作模型(VLA)逐Token解码导致规划延迟超1秒的瓶颈,arXiv论文提出SuperGlance架构。该方法挖掘VLM在生成首Token前的隐状态表征,将其作为编码器特征输入线性头,对固定的5273条候选轨迹进行评分。实验显示,该模型GPU规划耗时仅84ms,相比Qwen-Drive-1.0提速6倍,且在NAVSIM、PAI-AV及WOD-E2E基准上保持同等性能水平,验证了单次前向传播实现高质量驾驶决策的可行性。

技术层面,SuperGlance通过离线计算奖励加权监督目标与强化学习最大化期望奖励,并引入KL散度防止训练漂移。为保留视觉理解能力,采用抗捷径的多选题训练机制,确保模型在规划同时仍能准确回答场景问题。AlpaSim闭环测试表明,在300ms延迟内系统每故障里程数稳定;当计入GPU推理时间作为系统延迟时,其领先优势从2.8倍扩大至3.9倍,仅在延迟达1.2s时性能才低于基线模型。

关键看点 ​

  • 架构创新:利用VLM隐状态替代自回归解码,通过线性头对5273条固定候选轨迹评分,将规划转化为编码器式单次前向传播任务。
  • 推理性能:GPU规划耗时压缩至84ms,较Qwen-Drive-1.0快6倍;计入推理延迟后,闭环测试中的有效性能领先幅度从2.8倍提升至3.9倍。
  • 训练策略:结合离线奖励加权监督学习与RL优化,引入KL项防漂移;采用双选项顺序多选题训练消除捷径,兼顾驾驶策略与视觉问答能力。
  • 鲁棒性验证:AlpaSim闭环测试显示,在300ms系统延迟内每故障里程指标保持稳定,证明低延迟推理对提升端到端自动驾驶实际部署价值显著。

工程与产业观察 ​

该工作证明了VLM隐状态蕴含足够的空间语义信息用于实时控制,打破了“强推理必高延迟”的固有认知。对于具身智能开发者,这提供了一种将System 2大模型蒸馏为System 1实时策略的工程路径:通过固定轨迹集与离线奖励预计算,可将VLA推理耗时降至百毫秒级,解决了大模型在机器人高频控制回路中难以落地的核心痛点。

英文摘要 (Abstract) ​

Autonomous driving is turning to vision-language-action models (VLAs). They answer and plan by decoding, one step at a time. Each step is another forward pass, so a plan can take over a second of GPU time. Embodied AI often pairs such a slow, reasoning System 2 with a fast System 1 that acts. Human drivers plan at a glance, with System 1. Faster designs usually put System 1 outside the vision-language model (VLM), in a planner without its abilities. Yet before a VLM writes its first token, it has already read the scene into one hidden state. We show that, once trained, this hidden state is enough to drive and to answer questions about what the model sees. SuperGlance reads it to turn a decoder-only VLM into an encoder-style driving policy. Its linear head scores a fixed set of 5,273 candidate trajectories. With the set fixed, rewards are computed offline, so supervised targets can be reward-weighted and reinforcement learning maximizes the exact expected reward. Before and during policy training, multiple-choice training keeps the VLM's visual abilities in one pass, so the policy still answers questions about driving scenes. Training questions are built against shortcuts. Each must be answered right in two option orders. Against drift, a KL term keeps each stage near its starting model. One pass offers VLAs a more plausible path to System 1. SuperGlance answers visual multiple-choice questions as well as its base VLM does by decoding. It plans in 84 ms of GPU time, six times faster than Qwen-Drive-1.0, and at least as well on NAVSIM, PAI-AV and WOD-E2E. In AlpaSim closed loop, km per at-fault incident holds for both planners up to a 300-ms delay and drops beyond. With GPU time charged as delay, SuperGlance's lead over Qwen-Drive-1.0 grows from 2.8 to 3.9 times. Delayed by 1.2 s, our model falls below Qwen-Drive-1.0 at its own 518 ms.

作者:Jieyuan Pei, Xin Yan, Mingwei Xu, Haiyi Zhang, Siyu Zou, Zhangyi Hu, Zhuoxuan Li, Yuguang Yang, Rongjunchen Zhang, Cheng Chi, Hui Xiong, Yan Wang

基于本页技术方向关联的整机参数、核心零部件、实测报告与同赛道选型横评。

外部来源

本页事实对应的公开页面,以原文为准。(点击将离开本站)

  1. arXiv 2611.19138:SuperGlance: Driving at a Glance with a Vision-Language Model ↗访问 2026-11-18