# RoboTok：用于互联网演示视频检索与灵巧操作学习的可扩展数据引擎

> 原文链接：https://robotics.hellobike.com/papers/2026-09-02-2609-03199
> 频道：papers | 更新日期：2026-09-02 | 标签：arxiv, cs.RO
> 核心摘要：RoboTok用查询人类操作视频检索互联网演示，学习灵巧机器人策略，并在检索、下游策略和真机实验中验证。

# RoboTok：用于互联网演示视频检索与灵巧操作学习的可扩展数据引擎

::: v-pre
> 原文链接：[RoboTok: A Scalable Data Engine for Internet Demonstration Video Retrieval and Dexterous Manipulation Learning](https://arxiv.org/abs/2609.03199)（arXiv 2609.03199 · 发布日期：2026-09-02）

**RoboTok: A Scalable Data Engine for Internet Demonstration Video Retrieval and Dexterous Manipulation Learning**

RoboTok用查询人类操作视频检索互联网演示，学习灵巧机器人策略，并在检索、下游策略和真机实验中验证。

## 核心要点

- 机器人学习需广泛多样演示，但机器人数据采集昂贵且难以扩展到现实任务。
- 方法从3D手部轨迹学习以演员为中心参考系表达的潜在运动空间。
- 该表征可跨视角、场景外观和遮挡比较操作行为，并支持高效索引检索。
- 相比现有机器人数据检索方法，RoboTok检索更相关演示并提升下游成功率。

**验证范围与局限**：验证范围包括检索指标、下游机器人策略性能和真实机器人实验；摘要未说明具体任务或机器人数量。

## 英文摘要 (Abstract)

Robot learning increasingly depends on broad and diverse demonstrations, yet collecting robot data remains expensive and difficult to scale across the wide range of real-world tasks. To address this bottleneck, we introduce RoboTok, a scalable data engine that uses a query human manipulation video to retrieve manipulation-relevant internet demonstrations for training dexterous robot policies. Specifically, we learn a latent motion space from 3D hand trajectories expressed in estimated actor-centered reference frames. This representation enables manipulation behaviors to be compared across variations in camera viewpoint, scene appearance, and actor occlusions, while remaining compact enough for efficient indexing and retrieval over internet video collections. We evaluate RoboTok against existing robot-data retrieval approaches using retrieval metrics and downstream robot policy performance, showing that RoboTok retrieves more manipulation-relevant demonstrations and improves downstream task success. In real-world robot experiments, RoboTok-guided policies achieve a mean success improvement of 42.2 percentage points over the strongest baseline for each task, establishing hand-pose trajectory-aware retrieval as a scalable way to leverage continuously growing web video for robot learning.

作者：Howard H. Qian, Yiting Chen, Yunfei Xie, Kejia Ren, Podshara Chanrungmaneekul, Gaotian Wang, Bowen Wen, Chen Wei, Kaiyu Hang
:::

## 原始参考来源 (Sources)

- [arXiv 2609.03199：RoboTok: A Scalable Data Engine for Internet Demonstration Video Retrieval and Dexterous Manipulation Learning](https://arxiv.org/abs/2609.03199)