论文arXiv 2611.173946 天前
Agentic-Teacher:失败驱动VLA后训练数据引擎提升策略成功率
本文由 AI 根据公开内容整理,可能有误,请以原文为准:Agentic-Teacher: An Agentic Data Engine for Failure-Driven VLA Post-Training(arXiv 2611.17394)
核心内容
针对预训练视觉-语言-动作(VLA)模型后训练中有效监督数据匮乏的问题,研究提出Agentic-Teacher自主数据引擎。该系统接收VLA检查点、任务规范及环境工具,自动分析失败Rollout并选择干预状态,迭代开发闭环代码策略。验证通过的策略作为可复用演示生成器,专门采集VLA受挫状态下的成功续行轨迹,将模型弱点直接转化为针对性训练数据,实现从失败分析到数据生成的闭环。
为确保生成数据适配学习,引擎引入时间相关动作噪声以扩大状态-动作覆盖,剔除空闲帧同时保留夹爪关键转换,并利用冻结基座模型的Flow-matching Loss过滤低质轨迹。最终将筛选后的演示与原始成功轨迹混合进行后训练,使更新后的策略脱离代码智能体辅助独立运行。在仿真测试中,该方法将π0模型在LIBERO-10上的成功率从79.4%提升至87.8%,MetaWorld-10从80.4%升至90.6%,RoboCasa三个任务从0%突破至30.0%。
关键看点
- 失败驱动机制:自动解析VLA失败轨迹并定位干预状态,利用代码智能体迭代生成闭环修复策略,将模型特定弱点转化为定向数据采集源
- 数据清洗管线:引入时间相关动作噪声拓宽状态覆盖,剔除冗余空闲帧但保留夹爪状态跳变,确保合成演示的时序有效性与物理真实性
- 质量过滤准则:采用冻结基座VLA模型的Flow-matching Loss作为轨迹筛选指标,仅保留低损失样本参与后训练,防止噪声数据污染策略优化
- 仿真性能增益:在LIBERO-10和MetaWorld-10基准上分别将π0成功率提升8.4%和10.2%,并在RoboCasa三项零成功率任务中实现30%突破
工程与产业观察
该工作确立了“以模型失败指导数据合成”的VLA后训练范式,解决了通用演示数据对长尾错误覆盖不足的工程痛点。其基于Flow-matching Loss的数据清洗管线与代码智能体闭环生成机制,为具身大模型从仿真到真机的持续迭代提供了可复用的自动化数据基础设施参考。
英文摘要 (Abstract)
Improving a pretrained vision-language-action (VLA) model requires demonstrations that address its weaknesses and provide effective supervision for learning. Coding agents can autonomously generate demonstrations through environment interaction, yet successful execution alone does not ensure effective supervision for VLA learning. We introduce Agentic-Teacher, an autonomous data engine for failure-driven VLA post-training. Given a VLA checkpoint, a task specification, and configured environment tools, the engine analyzes failed rollouts, selects intervention states, and iteratively develops closed-loop code policies through execution feedback. Validated policies become reusable demonstration generators that collect successful continuations from states where the VLA struggles. To make these demonstrations suitable for learning, Agentic-Teacher introduces temporally correlated action noise to broaden state-action coverage, removes idle frames while preserving gripper transitions, and filters trajectories using the frozen base model's flow-matching loss. The resulting demonstrations are combined with successful VLA rollouts for post-training, yielding an updated policy that operates without coding-agent assistance. Agentic-Teacher thus connects the weaknesses of a target VLA to both the generation and curation of its training data. In simulation, it improves $\pi_0$ success rates from 79.4% to 87.8% on LIBERO-10, from 80.4% to 90.6% on MetaWorld-10, and from 0% to 30.0% across three RoboCasa tasks.
作者:Gongyue Teng, Yuqing Xie, Yixian Zhang, Feng Gao, Shu'ang Yu, Jiaji Rao, Sukai Wang, Guanghui Ren, Ya Zhang, Maoqing Yao, Yu Wang, Chao Yu, Xinlei Chen
外部来源
本页事实对应的公开页面,以原文为准。(点击将离开本站)