外观
论文arXiv 2609.204141 个月前
TouchSight：通过生成式视觉增强从第一视角视频进行裸手触觉预测
原文链接：TouchSight: Bare-Handed Tactile Prediction from Egocentric Video via Generative Visual Augmentation（arXiv 2609.20414 · 发布日期：2026-09-17）
TouchSight: Bare-Handed Tactile Prediction from Egocentric Video via Generative Visual Augmentation
TouchSight用单目第一视角视频与生成式裸手增强预测全手密集接触力，并在OakInk2和未见自然裸手视频上验证。
核心要点
- 触觉采集需接触界面测量，设备侵入、昂贵且限制大规模数据收集。
- 框架利用500小时压力手套记录和大量HOI数据，预测全手密集接触力。
- TwinTouch-20H用生成视频把戴手套画面重渲染为裸手并保留触觉标签。
- 在OakInk2优于既有接触预测方法，对未见裸手第一视角视频有定性泛化。
验证范围与局限：验证范围包括戴手套和生成裸手视频、OakInk2，以及未见数据集的自然裸手第一视角视频。
英文摘要 (Abstract)
Tactile signals provide direct contact and force measurements that are essential for understanding physical interactions and enabling dexterous robotic manipulation. However, tactile sensing requires direct measurement at contact interfaces, making large-scale data collection reliant on intrusive, costly, and restrictive instrumentation. We present TouchSight, a monocular egocentric vision framework for dense full-hand contact force prediction that leverages 500 hours of pressure-glove recordings and extensive hand-object interaction (HOI) data. To address the appearance gap between gloved training data and bare-hand real-world scenarios, we construct TwinTouch-20H: 20 hours of paired visual data in which generative video models re-render gloved recordings as bare-hand observations against new backgrounds while preserving the original measured tactile labels. TouchSight predicts dense force from both gloved and generated bare-hand videos, outperforms prior contact prediction methods on OakInk2, qualitatively generalizes to natural bare-hand egocentric videos from unseen datasets, and improves consistently as glove supervision scales. These results demonstrate that dense tactile signals can be recovered from egocentric vision alone, without tactile instrumentation at capture time.
作者：Danyan Zhou, Jinxuan Lu, Jiawei Lin, Tianxing Chen, Chuqiao Lyu, Wenbo Ding
来源
本页事实对应的公开页面，以原文为准。