# Dex-X：通过仿真交互从人类视频学习视觉-触觉灵巧操作

> 原文链接：https://robotics.hellobike.com/papers/2026-09-07-2609-07747
> 频道：papers | 更新日期：2026-09-07 | 标签：arxiv, cs.RO
> 核心摘要：DEX-X将单目人类演示重建到仿真中补全触觉监督，训练视觉-触觉策略，并在仿真与真机灵巧手臂任务上验证。

# Dex-X：通过仿真交互从人类视频学习视觉-触觉灵巧操作

::: v-pre
> 原文链接：[Dex-X: Learning Visual-Tactile Dexterous Manipulation From Human Videos with Simulated Interaction](https://arxiv.org/abs/2609.07747)（arXiv 2609.07747 · 发布日期：2026-09-07）

**Dex-X: Learning Visual-Tactile Dexterous Manipulation From Human Videos with Simulated Interaction**

DEX-X将单目人类演示重建到仿真中补全触觉监督，训练视觉-触觉策略，并在仿真与真机灵巧手臂任务上验证。

## 核心要点

- 人类视频包含丰富灵巧操作，但缺少接触交互所需的触觉信息。
- DEX-X在仿真中重建手-物交互，用物理接触动态提供触觉监督。
- 方法将策略蒸馏为使用点云观测和触觉传感的可部署策略。
- 教师策略仿真平均成功率65.9%；真机立方体抓取93%、桌面清洁53%。

**验证范围与局限**：验证范围为仿真六类任务，以及灵巧手臂平台上的多种抓取和接触丰富工具使用任务。

## 英文摘要 (Abstract)

Human videos are an abundant source of dexterous manipulation behaviors, but they lack tactile information that is crucial for contact-rich interaction. This raises a fundamental question: can robots learn deployable visual-tactile dexterous manipulation policies from human video demonstrations without robot-side data collection? We present DEX-X, a framework for learning visual-tactile dexterous manipulation from human videos through simulation. Our key insight is that simulation can serve as a tactile completion engine. Given monocular human demonstrations, DEX-X reconstructs hand-object interactions in simulation, where physically grounded contact dynamics provide tactile supervision unavailable in the original videos. Leveraging this recovered tactile information, we train visual-tactile dexterous manipulation policies and distill them into deployable policies operating on point-cloud observations and tactile sensing. We demonstrate zero-shot sim-to-real transfer on a dexterous hand-arm platform across diverse grasping and contact-rich tool-use tasks. The teacher policy achieves 65.9% average success across six task categories in simulation, while the distilled visual-tactile policy achieves 93% success on real-world cube picking and 53% on the challenging table-cleaning task. Zero-shot generalization to unseen object geometries is also observed on object-picking tasks. Our results suggest that simulated interaction is a key bridge between human videos and deployable dexterous manipulation policies, providing the missing physical supervision needed for scalable robot skill learning from Internet-scale human video data.

作者：Ruoqu Chen, Feixiang Ruan, Liu Cao, Zihao Wang, Botian Xu, Shiqin Tong, Jiajun Liu, Mingzhi Pei, Chenyu Zhang, Wanli Xing
:::

## 原始参考来源 (Sources)

- [arXiv 2609.07747：Dex-X: Learning Visual-Tactile Dexterous Manipulation From Human Videos with Simulated Interaction](https://arxiv.org/abs/2609.07747)