外观
论文arXiv 2610.1028821 天前
TouchScale：用于视觉-触觉学习的500小时人类视觉与触觉数据
原文链接：TouchScale: 500 Hours of Human Vision and Touch for Visual-Tactile Learning（arXiv 2610.10288 · 发布日期：2026-10-07）
TouchScale: 500 Hours of Human Vision and Touch for Visual-Tactile Learning
针对视频缺少接触与压力监督，TouchScale以统一可穿戴设备采集500小时人类视觉触觉数据，并在感知与机器人操作上验证收益。
核心要点
- 视频难记录物理接触与压力，既有触觉数据规模较小且常混合传感器或标注流程。
- TouchScale用统一可穿戴设置采集接触丰富交互，同步RGB-D、自我视角与双手触觉数据。
- 全量训练提升未见传感器零样本接触IoU，视觉预训练在三个动作识别基准中最好。
- 用于机器人策略视觉触觉中训练后，四项真实接触丰富操作任务平均成功率提高；数据将公开发布。
验证范围与局限：验证范围包括未见触觉传感器数据、三个动作识别基准和四项真实接触丰富操作任务。
英文摘要 (Abstract)
Large-scale egocentric human interaction data is becoming an important source of physical supervision for embodied learning, yet video alone leaves the contact and pressure that characterize physical interaction unrecorded. Recent visual-tactile datasets provide this missing supervision, but their synchronized tactile data remain far smaller in volume than human video. Moreover, the largest resources often merge recordings from different sensors or annotation procedures, which makes the effect of data scale difficult to isolate. We therefore introduce TouchScale, a 500-hour dataset of contact-rich human interaction recorded with a single unified wearable setup. Its approximately 2K predefined task descriptions span everyday activities and structured manipulation, and each recording temporally aligns egocentric RGB-D video with wrist RGB video and dense full-hand bimanual tactile measurements. Compared with prior tactile data, training on the full TouchScale raises zero-shot contact IoU on data from an unseen tactile sensor from 0.134 to 0.383. Pretraining a visual encoder on TouchScale also yields the highest action recognition accuracy on three benchmarks among the compared visual-tactile datasets. Used for visual-tactile mid-training of a robot policy, TouchScale improves the average real-world success rate across four contact-rich manipulation tasks from 22.5% to 57.5%. With the sensor and collection protocol held fixed, both zero-shot tactile prediction and robot success show an overall upward trend as more TouchScale data is used. These results suggest that human visual-tactile data collected at scale with consistent sensing benefits both perception and robot manipulation. We will publicly release TouchScale, including all synchronized visual-tactile recordings and reconstructed object models, to support future research on scalable visual-tactile learning.
作者：Dayou Li, Hao Wang, Qianqian Yang, Zihao Zhu, Haoquan Fang, Ziyao Zeng, Yan Han, Zihan Wang, Yan Wang, Baoru Huang
来源
本页事实对应的公开页面，以原文为准。