论文arXiv 2611.252432 天前
Verify2Act框架:潜空间验证使机械臂语言操作成功率达83.3%
本文由 AI 根据公开内容整理,可能有误,请以原文为准:Verify2Act: Critic-Guided Latent World Models for Verifying Language-Conditioned Manipulation Plans(arXiv 2611.25243)
核心内容
针对视觉语言模型(VLM)生成多步操作计划时缺乏场景变化接地模型、像素级世界模型推理慢且长程漂移的痛点,研究团队提出Verify2Act模块化验证框架。该系统将语义动作提案与潜在物理验证解耦,在低成本机械臂上仅使用数字孪生数据训练世界模型与Critic。实测显示,在6项语言条件操作任务中成功率达83.3%,显著优于无架构改进的潜空间模型(61.7%)、纯VLM(53.3%)及像素扩散模型(46.7%)。
技术上,V2A-WM基于Flow-matching扩展,引入Cross-attention动作接地与因果时序历史窗口,在DINOv2特征空间想象计划;双头Critic在执行前检查想象轨迹的时序一致性与目标满足度。系统实现了执行计划的100%验证覆盖,所有失败均源于底层抓取滑脱而非规划错误。在CALVIN仿真基准中,其五任务链成功率居首,且反思式VLM调用次数仅为像素扩散验证方法的1/30,大幅降低推理开销。
关键看点
- 架构创新:采用V2A-WM潜空间世界模型配合双头Critic,在DINOv2特征空间进行物理验证,实现语义提案与执行校验解耦
- 性能跃升:低成本机械臂6项语言操作任务成功率83.3%,较纯VLM基线提升30%,较像素扩散世界模型提升36.6%
- 零规划致错:系统对执行计划实现100%验证覆盖,所有失败案例均为底层抓取或放置滑脱,彻底消除高层规划逻辑错误
- 推理高效:CALVIN仿真五任务链成功率最优,反思式VLM调用量仅为像素扩散验证方法的1/30,显著降低长程任务算力成本
工程与产业观察
该工作验证了“语义提案+潜空间物理验证”范式的有效性,解决了VLM落地具身操作时的幻觉与长程漂移难题。仅依赖数字孪生数据训练即可在真机实现高成功率与零规划致错,为低成本机械臂接入大模型提供了可复现的工程路径,对缓解Sim-to-Real Gap及降低端侧推理负载具有重要参考价值。
英文摘要 (Abstract)
Vision-Language Models (VLMs) can propose multi-step manipulation plans, but they have no grounded model of how a scene will change, and the pixel-level world models that could check those plans are slow and drift over long horizons. We present Verify2Act, a modular verification framework that separates semantic action proposal from latent physical verification. A VLM proposes candidate plans, a latent world model (V2A-WM) imagines each of them in the DINOv2 feature space, and a dual-head critic checks the imagined rollouts for temporal consistency and goal satisfaction before anything is executed. V2A-WM extends recent flow-matching world models with cross-attention action grounding and a causal temporal history window. We deploy the full system on a low-cost robot arm, training the world model and critic only on data generated in a digital twin. Across six language-conditioned tasks, Verify2Act reaches 83.3% task success, compared with 61.7% for a latent world model without our architectural changes, 53.3% for the VLM alone, and 46.7% for a pixel-diffusion world model. It verified every plan it executed and incurred no planning-induced failures: all of its failed episodes were caused by low-level grasp or placement slips. In simulation, Verify2Act obtains the highest five-task chain success on CALVIN among the compared baselines with 30$\times$ fewer reflective VLM calls than pixel-diffusion verification, and the highest success on cluttered nut assembly.
作者:Chrisantus Eze, Christopher Crick
外部来源
本页事实对应的公开页面,以原文为准。(点击将离开本站)