Skip to content

论文arXiv 2611.176196 天前

TouchTwin:触觉真机到仿真闭环解决灵巧手力控缺失 ​

本文由 AI 根据公开内容整理,可能有误,请以原文为准:TouchTwin: Scaling Human Touch Data via Real-to-Sim-to-Real(arXiv 2611.17619)

核心内容 ​

针对人类演示数据驱动灵巧操作中“视觉可恢复姿态但丢失力觉”的核心缺陷,研究提出TouchTwin框架。实验显示仅靠视觉重建的策略抓取力仅为真值的0.31倍,物理重放则过估计至1.55倍。该工作通过引入实测触觉数据填补这一Force Gap,使仿真训练策略在保持测试集上抓取力达到记录真值的0.99倍,并在20次保持测试及36次未见扰动中分别取得20/20和35/36的成功率。

系统采用头部与腕部RGB-D相机配合880-taxel触觉手套采集TouchScale数据,并在MuJoCo中重建交互场景。Coding Agents自动生成经物理测试的物体程序,结合逐被试触觉手模型,以记录触觉设定各指接触与闭合力进行回放。相比纯视觉回放32%的接触帧漏检率,触觉接地回放的漏检率降至13%。此外,每次回放还生成带仿真接触与力标签的扰动演示,并渲染为新视角视频以扩充训练数据。

关键看点 ​

  • 力控精度对齐:触觉接地回放策略抓取力达真值0.99倍,显著优于纯视觉模仿的0.31倍与物理重放的1.55倍
  • 接触感知增强:基于触觉数据的接触预测器漏检率仅13%,较纯视觉回放32%的漏检率提升超一倍
  • 高保真数据采集:集成头腕RGB-D与880-taxel触觉手套,精确捕获手指闭合力与接触时序等视觉盲区信息
  • 自动化仿真重建:Coding Agents生成物理验证物体代码,结合触觉手模型实现带力标签的交互回放与数据增强

工程与产业观察 ​

该研究量化证实了纯视觉模仿学习在灵巧操作中存在本质力觉缺失,确立了高密度触觉反馈作为Sim-to-Real关键锚点的工程价值。其880-taxel手套采集与MuJoCo触觉接地回放范式,为具身智能从“看动作”向“感交互”演进提供了可复现的数据管线参考,提示硬件研发需重视触觉传感与仿真器的力觉对齐。

英文摘要 (Abstract) ​

Human demonstrations are becoming the main source of data for dexterous manipulation, and real-to-sim-to-real pipelines multiply each demonstration in simulation. These pipelines rebuild the interaction from vision, which recovers how the hand moves. Which fingers touch, when, and how hard remain unobserved, because a hand pose shows where each finger stops on the surface rather than how far it is driven to close. This gap is a force gap: on held-out recordings, a hand policy that imitates the observed pose grips with 0.31x the recorded force, and a policy trained on physics replays of vision-tracked poses grips with 1.55x. TouchTwin closes the gap with measured human touch. Each selected TouchScale recording, captured with head and wrist RGB-D cameras and an 880-taxel glove, is rebuilt in MuJoCo. Coding agents write the objects as physics-tested programs, and a per-subject tactile hand replays the interaction with the recorded touch setting the contact and the closing force of each finger. In simulation, a policy trained on touch-grounded replays grips at 0.99x the recorded force and succeeds in 20/20 trials on held-out recordings and 35/36 unseen perturbations; a contact predictor trained on them misses 13% of real contact frames, against 32% for vision-based replays. Each replay also seeds perturbed demonstrations with simulated contact and force labels, rendered as realistic video in new views and scenes.

作者:Huiwen Han, Nuo Chen, Ziyao Zeng, Dayou Li, Hao Wang, Amir Hossein Rahmati, Xiaoning Qian, Bangya Liu, Zhiwen Fan

基于本页技术方向关联的整机参数、核心零部件、实测报告与同赛道选型横评。

外部来源

本页事实对应的公开页面,以原文为准。(点击将离开本站)

  1. arXiv 2611.17619:TouchTwin: Scaling Human Touch Data via Real-to-Sim-to-Real ↗访问 2026-11-16