Skip to content

论文arXiv 2611.0869813 天前

VICTR框架以价值感知检索增强VLA模型上下文学习能力 ​

本文由 AI 根据公开内容整理,可能有误,请以原文为准:VICTR: Value-Informed In-Context Learning for Vision-Language-Action Models(arXiv 2611.08698)

核心内容 ​

针对视觉-语言-动作(VLA)模型在上下文学习(ICL)中仅依赖视觉相似性检索导致执行阶段错配的问题,研究团队提出VICTR框架。该方法引入上下文价值函数估计器(IC-VFE),通过结合参考演示视频、语言指令与当前观测来估计任务相对进度,从而精准匹配执行阶段。在未见过的域外真实操作任务测试中,VICTR将微调版 $\pi_{0.5}$ 基线的成功率从6.2%显著提升至23.8%,任务进度率由28.9%增至55.5%。

技术实现上,IC-VFE利用参考演示对齐当前状态与执行阶段,即使面对训练未见的任务也能有效工作。系统根据估计的进度值检索对应阶段的演示片段,并将其作为上下文输入给经过后训练的 $\pi_{0.5}$ 策略模型进行动作生成。该方案解决了传统检索增强生成(RAG)在机器人长时序操作中的时序对齐难题,相关代码、数据集及演示视频已在项目主页开源,为具身智能免微调适配提供了新范式。

关键看点 ​

  • 核心架构:提出VICTR框架与IC-VFE估计器,利用参考视频、指令及观测联合估计任务进度,替代纯视觉相似度检索以解决执行阶段错配。
  • 性能跃升:在未见过的域外真实操作任务中,将 $\pi_{0.5}$ 基线成功率从6.2%提升至23.8%,任务进度率从28.9%翻倍至55.5%。
  • 策略集成:基于进度估计动态检索对应阶段演示块,供给经后训练的 $\pi_{0.5}$ 策略模型,实现无需额外微调的上下文动作生成。
  • 开源资产:项目主页已开放完整代码、数据集及实验视频,支持社区复现与二次开发,验证了价值感知检索在具身ICL中的有效性。

工程与产业观察 ​

VICTR证明了在VLA模型中引入显式进度估计可大幅改善上下文学习的检索质量,突破了纯视觉匹配的局限。对于具身算法工程师而言,这种“价值感知+动态检索”范式为解决长时序任务泛化提供了低成本路径,且基于 $\pi_{0.5}$ 的改进表明现有基础模型仍有巨大后训练优化空间,值得在自有本体上验证迁移效果。

英文摘要 (Abstract) ​

Recent advances in robot foundation models have highlighted the potential of in-context learning (ICL) for adapting robot behavior from demonstrations without further fine-tuning. Previous work has used retrieval-based post-training to equip pretrained vision-language-action (VLA) models with the capability for ICL by teaching them to condition action predictions on retrieved visually-relevant demonstration chunks. However, the effectiveness of retrieval hinges on selecting context relevant to the current execution stage, which visual similarity alone may not capture. We introduce VICTR, a framework enabling in-context learning in VLAs via progress-informed context retrieval built around an In-Context Value Function Estimator (IC-VFE). IC-VFE conditions on a reference demonstration video, a language instruction and the current observation to estimate task-relative progress. By conditioning progress estimation on a reference demonstration, IC-VFE can align the current state with a demonstrated execution even for tasks unseen during training. VICTR uses the estimated progress to select demonstration chunks at corresponding stages of execution and supplies them to a $\pi_{0.5}$ policy post-trained for in-context action generation. On unseen out-of-domain real-world manipulation tasks, VICTR raises the success rate from 6.2% for a fine-tuned $\pi_{0.5}$ baseline to 23.8% and more than doubles the task progress rate, from 28.9% to 55.5%. Project page with code, data and videos: https://ai4ce.github.io/VICTR/

作者:Luca Macesanu, Tianluo Zhang, Aditya Patil, Archit Sharma, Kevin Angers, Christopher M. Clark, Chen Feng, Juexiao Zhang

基于本页技术方向关联的整机参数、核心零部件、实测报告与同赛道选型横评。

外部来源

本页事实对应的公开页面,以原文为准。(点击将离开本站)

  1. arXiv 2611.08698:VICTR: Value-Informed In-Context Learning for Vision-Language-Action Models ↗访问 2026-11-09