外观
论文arXiv 2608.150602 个月前
EgoTac：从第一人称视觉进行野外触觉预测
原文链接：EgoTac: In-the-wild Tactile Prediction from Egocentric Vision（arXiv 2608.15060 · 发布日期：2026-08-15）
EgoTac: In-the-wild Tactile Prediction from Egocentric Vision
EgoTac从第一人称人类视频直接预测触觉信息，并在域内、域外基准及真实视频上验证。
核心要点
- 多数第一人称人类数据缺少触觉，大规模直接采集受传感器限制。
- EgoTac在统一图像-触觉语料上训练，预测连续力测量和二值接触。
- 实验显示域内平均力误差低于0.06N，域外接触预测优于现有方法。
- 模型可捕捉真实触觉升降模式，并支持无约束真实视频零样本预测。
验证范围与局限：验证范围包括域内力预测、域外接触预测基准和无约束真实世界视频零样本预测。
英文摘要 (Abstract)
Touch is fundamental to dexterous manipulation, yet most egocentric human data increasingly used for robot learning lacks tactile information. Directly collecting large-scale tactile data is challenging due to sensor limitations, while human video data is abundant, contact-rich, and easily scalable. This motivates a natural question: can tactile signals be inferred purely from vision? To address this, we introduce EgoTac, a generalizable model that predicts rich tactile information directly from egocentric human videos. EgoTac is trained on a unified corpus of over 5.7M image-tactile pairs, covering both continuous force measurements and binary contacts. By learning from this diverse dataset, EgoTac captures nuanced touch dynamics across varied interactions. Experiments demonstrate strong performance: in-domain prediction achieves an average force error below 0.06N. On out-of-domain contact prediction benchmarks, EgoTac consistently outperforms the state-of-the-art contact estimator. It also captures the rise and fall patterns of real tactile data and enables zero-shot predictions on unconstrained real-world videos. Scaling analyses further reveal that both data diversity and volume improve performance steadily. Overall, EgoTac provides a scalable pathway to extract tactile priors from egocentric human videos, enabling broadly applicable tactile-aware robot learning. Related resources can be found at https://egotac.github.io/ .
作者：Wenkang Zhang, Chengbo Yuan, Zicheng Zhang, Zhengxue Cheng, Yang Gao
来源
本页事实对应的公开页面，以原文为准。