论文/arXiv 2610.22932·12 天前

Ego2Contact：从自我中心人类视频预测触觉用于富接触机器人操作

Ego2Contact从人类第一视角视频学习视觉-触觉先验，迁移到机器人手，并在人体、机器人数据及仿真/真机策略中验证。

AI 整理

• 硬件零部件
• 仿真与训练
• 数据与采集

原文链接：Ego2Contact: Predicting Touch from Egocentric Human Video for Contact-Rich Robot Manipulation（arXiv 2610.22932 · 发布日期：2026-10-16）

Ego2Contact: Predicting Touch from Egocentric Human Video for Contact-Rich Robot Manipulation

Ego2Contact从人类第一视角视频学习视觉-触觉先验，迁移到机器人手，并在人体、机器人数据及仿真/真机策略中验证。

核心要点

• 富接触操作需要接触位置和作用力，但纯视觉有歧义，机器人触觉数据因硬件与采集成本稀缺。
• 方法融合时序场景上下文与手部局部视觉特征，从RGB视频推断双手触觉场。
• 通过解剖区域对应，将不同机器人触觉布局对齐到规范网格，用于微调和策略输入。
• 在人体数据和适配后的机器人数据上优于既有触觉预测器；仿真中优于无触觉基线。

验证范围与局限：验证范围包括人体数据、适配后的机器人数据、仿真强化学习和真机策略后训练；真机效果摘要未给出具体结果。

英文摘要 (Abstract)

Contact locations and interaction forces are critical for contact-rich robot manipulation but are often ambiguous from vision alone. Measuring them on robots requires specialized tactile hardware and platform-specific data collection, which keeps robot tactile data scarce. Human egocentric video is far easier to scale but lacks touch. Recent predictors trained on egocentric recordings paired with tactile-glove measurements can predict this missing signal from video alone. These predictors, however, remain confined to human hands, which differ from robot hands in appearance and tactile layout. We present Ego2Contact, which learns a visual-tactile prior from human interactions and transfers it to robot hands. Its predictor fuses temporal scene context with hand-local visual features to infer bimanual tactile fields from the RGB video. Anatomical region correspondences align varying robot tactile layouts with the predictor's canonical grid, turning robot measurements into fine-tuning targets and predictions into policy inputs. Ego2Contact outperforms prior tactile predictors on human data, including unseen objects, and on robot data after adaptation. The adapted predictor provides tactile inputs for both simulated reinforcement learning and real-robot policy post-training. In simulation, policies using predicted touch outperform non-tactile baselines and recover part of the benefit of ground-truth tactile sensing. These results show that human-learned tactile priors can supply touch when direct tactile sensing is limited.

作者：Qianqian Yang, Dayou Li, Bangya Liu, Zihao Zhu, Nuo Chen, Siyuan Xu, Hezhen Hu, Chen Yu, Rebecca Friesen, Minghui Zheng

来源

本页事实对应的公开页面，以原文为准。

1. arXiv 2610.22932：Ego2Contact: Predicting Touch from Egocentric Human Video for Contact-Rich Robot Manipulation访问 2026-10-21