论文arXiv 2611.0904113 天前
DobotTrack:0.8B参数VLA框架实现77.9%具身跟踪成功率
本文由 AI 根据公开内容整理,可能有误,请以原文为准:DobotTrack: Target-Grounded Vision-Language-Action Learning for Embodied Person Tracking(arXiv 2611.09041)
核心内容
针对语言引导的具身人员跟踪难题,研究团队发布DobotTrack视觉-语言-动作(VLA)框架。该0.8B参数模型在EVT-Bench基准测试中取得77.9%平均成功率,较OmTrackVLA提升16.9个百分点。项目同步开源了基于四足机器人采集的实世界跟踪数据集及完整代码,为具身智能体在自我中心视角下的目标识别与持续跟随提供了端到端解决方案。
技术上采用Q-identity历史视觉压缩模块,利用目标接地预测选择性保留身份判别特征;结合人物分割、边界框接地、视觉空间分类等五项辅助任务增强定位精度。配套构建包含126个片段、596,323帧掩码标注的四足机器人实测数据集,通过掩码自动派生边界框与视觉空间标签,实现了无需额外人工标注的接地监督,有效支撑了从仿真到真机的策略迁移训练。
关键看点
- 性能指标:0.8B参数模型在EVT-Bench三项任务平均成功率达77.9%,超越OmTrackVLA基线16.9个百分点
- 核心架构:提出Q-identity模块,基于目标接地预测对历史视觉token进行条件压缩,保留身份与空间线索
- 数据资产:开源四足机器人实采数据集,含126个episode及59万帧掩码标注,支持自动派生接地标签
- 训练策略:集成人物分割、未来DINO特征预测等五项辅助任务,利用互补掩码与边界框监督增强目标定位
工程与产业观察
该工作验证了小参数量VLA模型通过高效时序建模与细粒度接地监督,可在具身跟踪任务上超越更大模型。其基于四足机器人的大规模掩码标注数据集与自动化标签派生管线,显著降低了真机数据采集成本,为具身感知算法从仿真走向实体部署提供了高价值的数据基础设施与工程范式参考。
英文摘要 (Abstract)
Language-guided embodied person tracking requires a mobile agent to identify a target from a referring description and consistently follow the target under egocentric observations. We present DobotTrack, a target-grounded vision-language-action (VLA) framework that integrates auxiliary grounding supervision with efficient temporal modeling. We systematically investigate five auxiliary tasks---person segmentation, bounding-box grounding, visual-spatial classification (VSC), future DINO feature prediction, and future-frame prediction---and adopt complementary mask and bounding-box supervision to enhance target delineation and localization. We introduce Q-identity, a target-conditioned historical visual compression module that leverages target-grounding predictions to selectively preserve identity-discriminative appearance features and spatial cues in compact historical tokens. On EVT-Bench, our 0.8B-parameter model achieves an average success rate of 77.9%, exceeding OmTrackVLA by 16.9 percentage points across the three tasks. We further construct a real-world person-tracking dataset collected using a quadruped robot, comprising 126 episodes and 596,323 mask-annotated frames. The target masks enable automatic derivation of bounding boxes and visual-spatial labels encoding horizontal position and apparent image scale, providing additional grounding supervision without separate manual annotation of these labels. Together, our framework and annotation pipeline support learning target-grounded tracking policies from simulated and real-world data. Code is available at https://github.com/Dobot-Team/dobot_track.
作者:Shengjun Deng, Yinghan Zhang, Chengrui Wei, Liu Liang, Songyuan Liu, Pengkun Zheng, Yajun Zheng, Ying Zhang, Yuan Tian, Xi Li, Teng Wang
外部来源
本页事实对应的公开页面,以原文为准。(点击将离开本站)