论文/arXiv 2610.09718·21 天前

YUBI-STAG：通过自动化视频-语言Grounding实现VLA的接触与语义丰富对齐

YUBI-STAG通过自动视频-语言grounding为操作演示生成交互语义，在YUBI-STAG-Bench和双手实验中对齐VLA细粒度指令。

AI 整理

• 本体 SDK 与控制
• 数据与采集
• 具身模型

原文链接：YUBI-STAG: Contact and Semantic-Rich Alignment for VLAs via Automated Video-Language Grounding（arXiv 2610.09718 · 发布日期：2026-10-07）

YUBI-STAG: Contact and Semantic-Rich Alignment for VLAs via Automated Video-Language Grounding

YUBI-STAG通过自动视频-语言grounding为操作演示生成交互语义，在YUBI-STAG-Bench和双手实验中对齐VLA细粒度指令。

核心要点

• 现有机器人演示多为粗粒度任务描述，缺少夹爪、接触对象、抓取与移动方式等执行细节。
• YUBI-STAG结合接触对象分割与视觉语言模型，标注物体身份、属性状态、单夹爪动作和空间交互。
• YUBI-VLM由YUBI-STAG蒸馏而来，可从原始未分段视频中以更少推理调用恢复动作结构。
• 实验显示YUBI-VLM保留较多标注准确率且运行更快，后训练VLA提升双手操作和指令跟随。

验证范围与局限：验证范围为YUBI-STAG-Bench的时序、语义、空间grounding任务及双手操作实验。

英文摘要 (Abstract)

Vision-Language-Action (VLA) models acquire broad manipulation capabilities via large-scale pretraining, yet eliciting them through language requires fine-grained alignment between instructions and physical interactions. Existing robot demonstrations typically provide only coarse task descriptions, omitting how actions are executed, including which gripper acts, which object is contacted, and how it is grasped and moved. We introduce YUBI-STAG, a framework for Spatio-Temporal Annotation and Grounding that automatically enriches manipulation demonstrations with interaction-rich semantics to align pretrained VLAs with fine-grained manipulation language. Combining contact-object segmentation with vision-language models, YUBI-STAG annotates object identities, attributes and states, per-gripper actions, bimanual coordination, and spatially grounded interactions. To address YUBI-STAG's reliance on localized sequences and multi-stage VLM inference, we distill it into YUBI-VLM. YUBI-VLM directly recovers action structure and annotations from raw, unsegmented video in few inference calls and operates from wrist views alone. We evaluate both frameworks on YUBI-STAG-Bench across temporal, semantic, and spatial grounding tasks. YUBI-VLM retains much of YUBI-STAG's annotation accuracy with fewer inference calls and shorter runtime while generalizing to unseen manipulations. Finally, post-training VLA policies on these annotations aligns them with fine-grained language and contact-aware structure. Bimanual experiments demonstrate improved performance and instruction following, including control over object identity, acting gripper, target location, and spatial relations absent from original labels.

作者：Masatoshi Tateno, Takehiko Ohkawa, Yueh-Hua Wu, Hanlong Li, Tatsuya Matsushima, Yoichi Sato, Kei Ota

来源

本页事实对应的公开页面，以原文为准。

1. arXiv 2610.09718：YUBI-STAG: Contact and Semantic-Rich Alignment for VLAs via Automated Video-Language Grounding访问 2026-10-21