# VisTacAlign：使用视觉-触觉对齐在触觉人类和机器人演示上协同训练灵巧策略

> 原文链接：https://robotics.hellobike.com/papers/2026-09-25-2609-30959
> 频道：papers | 更新日期：2026-09-25 | 标签：arxiv, cs.RO
> 核心摘要：VisTacAlign通过视觉与触觉对齐，将人类和机器人演示共同用于3D视觉触觉灵巧策略训练，并在三项真实任务上验证。

# VisTacAlign：使用视觉-触觉对齐在触觉人类和机器人演示上协同训练灵巧策略

::: v-pre
> 原文链接：[VisTacAlign: Co-Training Dexterous Policies on Tactile Human and Robot Demonstrations using Visual-Tactile Alignment](https://arxiv.org/abs/2609.30959)（arXiv 2609.30959 · 发布日期：2026-09-25）

**VisTacAlign: Co-Training Dexterous Policies on Tactile Human and Robot Demonstrations using Visual-Tactile Alignment**

VisTacAlign通过视觉与触觉对齐，将人类和机器人演示共同用于3D视觉触觉灵巧策略训练，并在三项真实任务上验证。

## 核心要点

- 人类演示成本低，但用于机器人策略共训时，需要弥合策略所用各模态的人机差距。
- 框架将手套跟踪的人手运动重定向到17-DoF触觉机器人手，并对双目视觉进行机器人手替换与点云重算。
- 电容触觉手套在信号空间对齐到机器人指尖传感器，扩散Transformer消费点云、本体和逐指触觉token。
- 在三项需精确力的真实任务中，加入对齐人类演示优于仅用机器人数据；消融显示触觉输入和视觉对齐均必要。

**验证范围与局限**：验证范围为三个真实世界任务：乐高装配、采摘不同大小草莓，以及启动并提起电钻。

## 英文摘要 (Abstract)

Human demonstrations are a cheap source of data for dexterous manipulation, but co-training a robot policy on them requires closing the human--robot gap in every modality the policy consumes. We present VisTacAlign, a framework for co-training 3D-visual-tactile dexterous policies on human and robot demonstrations. Glove-tracked human hand motion is retargeted to a 17-DoF tactile robot hand with a one-time fingertip correction. The human hand is then erased from both stereo views and replaced by a posed robot-hand mesh painted with pixels from robot recordings, and a real-time stereo foundation model is re-run on the composite, so the human point clouds carry the same stereo errors and visibility as the robot ones. Finally, a capacitive tactile glove is aligned to the robot's fingertip sensors in its signal space, giving one interpretable per-finger force representation. A diffusion transformer consumes point-cloud, proprioceptive, and per-finger tactile tokens. On three real-world tasks requiring precise force -- Lego assembly, plucking strawberries of varying size, and activating and lifting a power drill -- adding aligned human demonstrations to existing robot data improves over robot-only policies, and ablations show that both tactile input and visual alignment are necessary. Project page: https://vis-tac-align.github.io

作者：Julien Poffet, Matthew Strong, Ankush Dhawan, Baiyu Shi, Shalika Neelaveni, Yujia Yuan, Zhenan Bao, Monroe Kennedy
:::

## 原始参考来源 (Sources)

- [arXiv 2609.30959：VisTacAlign: Co-Training Dexterous Policies on Tactile Human and Robot Demonstrations using Visual-Tactile Alignment](https://arxiv.org/abs/2609.30959)