外观
论文HF Daily Papers · arXiv 2610.1078721 天前
NavGPT-3发布OS级导航运行时，RxR-CE成功率追平人类
本文由 AI 根据公开内容整理，可能有误，请以原文为准：NavGPT-3: Harnessing Context in a Hierarchical Navigation Runtime（HF Daily Papers · arXiv 2610.10787）
核心内容
NavGPT-3提出一种连接语言模型推理与底层动作策略的具身导航框架，在R2R-CE基准取得81.51 SR的SOTA成绩。在更具挑战性的RxR-CE测试中，该系统成功率达90.43%，首次匹配人类跟随者水平（90.4 SR），路径保真度78.47 nDTW亦优于人类的77.7 nDTW，且单回合平均耗时仅1分22秒，显著快于人类约3分钟的表现。
技术上采用类OS运行时架构，将推理、执行与监控解耦为独立线程并支持中断切换，使机器人能实时响应突发环境事件。底层NavGPT VLA基于1928万样本训练，利用codec分配机制按场景变化动态分配视觉token。该设计将系统最小反应时间从纯语言模型决策的3-19秒大幅降低至动作策略执行的0.5-1秒（1-2Hz），有效解决了高层语义推理与低层物理控制的时延鸿沟。
关键看点
- 人机对齐性能：RxR-CE成功率90.43%持平人类，nDTW达78.47超越人类基准，单回合耗时1分22秒较人类提速超50%
- OS级运行时：构建多线程调度架构支持上下文隔离与权限管理，通过中断机制实现推理与运动控制的动态切换
- VLA动态Token：8B参数模型经1928万样本训练，采用codec allocation按场景变化比例分配视觉token，R2R-CE SR达76.94
- 控制时延优化：动作策略接管执行后系统反应时间从3-19s降至0.5-1s，验证了具身接口对连接LLM与物理控制的关键作用
工程与产业观察
NavGPT-3证明了类OS运行时是弥合大模型长程推理与机器人实时控制的有效范式。通过线程级中断与VLA动态token分配，成功将高层语义决策转化为1-2Hz的低延迟物理响应，并在复杂导航基准上实现人机同效。该架构为具身智能体从仿真走向真实环境提供了关键的系统工程参考，即将开源的代码与模型将进一步加速社区在具身接口层面的技术迭代。
英文摘要 (Abstract)
Language models trained with long-horizon agentic reinforcement learning can generalize knowledge through reasoning, express precise actions, and pursue goals over many steps, raising the ceiling on what an embodied agent can understand and decide. Physical interaction, however, remains the domain of action policies, which provide dense, low-latency control. We present NavGPT-3, a harness that connects the two models, with an OS-like runtime built above it: reasoning, acting, and monitoring run as threads with their own context, tools, and permissions, while the runtime schedules them and decides which thread controls the robot's motion, so that the robot can react to sudden real-world events through interruption and thread switching. Beneath it, our action policy NavGPT VLA, trained on 19.28M examples, allocates visual tokens using codec allocation, in proportion to scene change; its 8B model alone reaches 76.94 SR on R2R-CE and leads RxR-CE with 82.39 SR. With the complete harness, NavGPT-3 sets the state of the art on R2R-CE (81.51 SR) and, for the first time, brings an autonomous agent to human level: on RxR-CE it matches human followers in success (90.43 vs. 90.4 SR) and path fidelity (78.47 vs. 77.7 nDTW) at 1 min 22 s per episode, versus roughly 3 min for a human. We comprehensively ablate the harness design and the interaction between the two models, showing how tools and the action policy shape the path from language-model reasoning to physical control: when NavGPT VLA executes the route, the reasoning loop shortens and the system's minimum reaction time falls from 3-19 s per language-model decision to 0.5-1 s per action-policy step (1-2 Hz). These results show that designing this embodied interface is central to connecting frontier language-model intelligence with low-level physical control. We will release all models, code, and evaluation records.
Hugging Face Daily Papers 热度：5 upvotes 作者：Gengze Zhou, Yicong Hong, Jiazhao Zhang, Xunyi Zhao, Jian Zhou, Zixing Lei, Zun Wang, Chongyang Zhao, Xionghui Chen, Stephen Gould, Anton van den Hengel, Qi Wu
来源
本页事实对应的公开页面，以原文为准。