# ZeroVLN：零SFT微调VLM实现连续环境视觉语言导航

> 原文链接：https://robotics.hellobike.com/papers/2026-11-19-2611-26414-335988aa
> 频道：papers | 更新日期：2026-11-19 | 标签：auto, arxiv, hf-daily-papers
> 核心摘要：提出ZeroVLN慢快架构，通过语义工具调用与闭环强化学习联合优化VLM决策与动作执行，在VLN-CE基准上显著优于传统SFT方法，且保持基座模型通用能力不变。

# ZeroVLN：零SFT微调VLM实现连续环境视觉语言导航

::: v-pre
> 本文由 AI 根据公开内容整理，可能有误，请以原文为准：[ZeroVLN: Learning to Navigate without Supervised Fine-Tuning of the VLM Brain](https://arxiv.org/abs/2611.26414)（HF Daily Papers · arXiv 2611.26414）

## 核心内容

针对视觉语言导航（VLN）中监督微调（SFT）导致VLM基座权重偏移及通用能力退化的问题，研究团队提出ZeroVLN框架。该方法首创无需导航域SFT的慢快系统架构，在标准VLN-CE连续环境基准测试中性能大幅超越代表性SFT分层方法，同时仅对VLM权重产生微小改动，有效保留了模型的通用视觉理解能力，成为首个在VLN-CE中实现零SFT且通过闭环智能体RL联合优化的框架。

技术上，ZeroVLN摒弃让VLM直接预测几何目标的传统范式，转而构建语义工具调用接口，使VLM以自然语言向动作专家传达移动决策。系统先适配动作专家执行指令，再通过闭环强化学习（RL）联合优化VLM的语义决策策略与动作专家的连续控制策略。这种解耦设计避免了端到端微调对大模型参数的破坏，验证了在具身导航任务中利用冻结基座模型进行高效策略对齐的工程可行性。

## 关键看点

- **零SFT架构**：首创无需导航域监督微调的慢快VLN框架，避免VLM基座权重偏移与通用视觉能力退化
- **语义工具接口**：VLM通过自然语言工具调用传达移动决策，替代传统几何目标预测，实现语义与运动解耦
- **闭环联合优化**：采用闭环强化学习同步训练VLM语义决策与动作专家连续执行策略，提升端到端导航效能
- **VLN-CE性能**：在连续环境基准上显著优于代表性SFT分层方法，验证了冻结基座模型在具身任务中的有效性

## 工程与产业观察

该工作为具身智能提供了保留大模型通用能力的导航新范式。通过语义接口解耦与闭环RL替代全量SFT，解决了VLM适配机器人任务时的灾难性遗忘痛点，对降低具身算法训练成本、复用多模态基座模型具有重要工程参考价值。

## 英文摘要 (Abstract)

Representative vision-language navigation (VLN) methods typically couple a vision-language model (VLM) brain with a learnable action head and fine-tune the VLM brain on navigation-domain data to predict geometric goals. However, such adaptation can substantially alter base VLM weights and degrade general visual capabilities. We introduce ZeroVLN, a slow-fast navigation framework that achieves effective navigation with zero navigation-domain SFT of the VLM brain. Rather than teaching the brain to predict geometric goals, we equip it with a semantic tool-calling interface for conveying movement decisions to an action expert in natural language. We first adapt the action expert to carry out these movement requests, then jointly improve the brain's movement decisions and the action expert's continuous execution through closed-loop reinforcement learning (RL). Trained on the same navigation episodes, ZeroVLN substantially outperforms representative SFT-based hierarchical VLN methods on standard VLN-CE (continuous environments) benchmarks, while inducing only minor changes to the brain's weights and largely preserving its general capabilities. To our knowledge, ZeroVLN is the first slow-fast VLN framework with zero navigation-domain SFT of the VLM brain, and the first to jointly optimize its slow and fast policies through closed-loop agentic RL in VLN-CE.

Hugging Face Daily Papers 热度：20 upvotes
作者：Zixuan Huang, YANG ZHOU, Guli Zhang, Sunzhu Li, Hongyan Xie, Zehao Chen, Fuzhen Zhuang, Kaixuan Wang, Xiaozhi Chen, Yikun Ban, Deqing Wang
:::

## 原始参考来源 (Sources)

- [HF Daily Papers · arXiv 2611.26414：ZeroVLN: Learning to Navigate without Supervised Fine-Tuning of the VLM Brain](https://arxiv.org/abs/2611.26414)