# TouchTwin：触觉真机到仿真闭环解决灵巧手力控缺失

> 原文链接：https://robotics.hellobike.com/papers/2026-11-16-2611-17619-8ff68ce0
> 频道：papers | 更新日期：2026-11-16 | 标签：auto, arxiv
> 核心摘要：针对视觉模仿学习中灵巧操作力控缺失痛点，TouchTwin利用880触点手套与MuJoCo构建触觉接地回放管线。策略抓取力达真值0.99倍，接触预测漏检率降至13%，验证了触觉数据在Sim-to-Real中的关键补全作用。

# TouchTwin：触觉真机到仿真闭环解决灵巧手力控缺失

::: v-pre
> 本文由 AI 根据公开内容整理，可能有误，请以原文为准：[TouchTwin: Scaling Human Touch Data via Real-to-Sim-to-Real](https://arxiv.org/abs/2611.17619)（arXiv 2611.17619）

## 核心内容

针对人类演示数据驱动灵巧操作中“视觉可恢复姿态但丢失力觉”的核心缺陷，研究提出TouchTwin框架。实验显示仅靠视觉重建的策略抓取力仅为真值的0.31倍，物理重放则过估计至1.55倍。该工作通过引入实测触觉数据填补这一Force Gap，使仿真训练策略在保持测试集上抓取力达到记录真值的0.99倍，并在20次保持测试及36次未见扰动中分别取得20/20和35/36的成功率。

系统采用头部与腕部RGB-D相机配合880-taxel触觉手套采集TouchScale数据，并在MuJoCo中重建交互场景。Coding Agents自动生成经物理测试的物体程序，结合逐被试触觉手模型，以记录触觉设定各指接触与闭合力进行回放。相比纯视觉回放32%的接触帧漏检率，触觉接地回放的漏检率降至13%。此外，每次回放还生成带仿真接触与力标签的扰动演示，并渲染为新视角视频以扩充训练数据。

## 关键看点

- **力控精度对齐**：触觉接地回放策略抓取力达真值0.99倍，显著优于纯视觉模仿的0.31倍与物理重放的1.55倍
- **接触感知增强**：基于触觉数据的接触预测器漏检率仅13%，较纯视觉回放32%的漏检率提升超一倍
- **高保真数据采集**：集成头腕RGB-D与880-taxel触觉手套，精确捕获手指闭合力与接触时序等视觉盲区信息
- **自动化仿真重建**：Coding Agents生成物理验证物体代码，结合触觉手模型实现带力标签的交互回放与数据增强

## 工程与产业观察

该研究量化证实了纯视觉模仿学习在灵巧操作中存在本质力觉缺失，确立了高密度触觉反馈作为Sim-to-Real关键锚点的工程价值。其880-taxel手套采集与MuJoCo触觉接地回放范式，为具身智能从“看动作”向“感交互”演进提供了可复现的数据管线参考，提示硬件研发需重视触觉传感与仿真器的力觉对齐。

## 英文摘要 (Abstract)

Human demonstrations are becoming the main source of data for dexterous manipulation, and real-to-sim-to-real pipelines multiply each demonstration in simulation. These pipelines rebuild the interaction from vision, which recovers how the hand moves. Which fingers touch, when, and how hard remain unobserved, because a hand pose shows where each finger stops on the surface rather than how far it is driven to close. This gap is a force gap: on held-out recordings, a hand policy that imitates the observed pose grips with 0.31x the recorded force, and a policy trained on physics replays of vision-tracked poses grips with 1.55x. TouchTwin closes the gap with measured human touch. Each selected TouchScale recording, captured with head and wrist RGB-D cameras and an 880-taxel glove, is rebuilt in MuJoCo. Coding agents write the objects as physics-tested programs, and a per-subject tactile hand replays the interaction with the recorded touch setting the contact and the closing force of each finger. In simulation, a policy trained on touch-grounded replays grips at 0.99x the recorded force and succeeds in 20/20 trials on held-out recordings and 35/36 unseen perturbations; a contact predictor trained on them misses 13% of real contact frames, against 32% for vision-based replays. Each replay also seeds perturbed demonstrations with simulated contact and force labels, rendered as realistic video in new views and scenes.

作者：Huiwen Han, Nuo Chen, Ziyao Zeng, Dayou Li, Hao Wang, Amir Hossein Rahmati, Xiaoning Qian, Bangya Liu, Zhiwen Fan
:::

## 原始参考来源 (Sources)

- [arXiv 2611.17619：TouchTwin: Scaling Human Touch Data via Real-to-Sim-to-Real](https://arxiv.org/abs/2611.17619)