# ME-Dex 1.0：将异构触觉感知引入世界动作建模

> 原文链接：https://robotics.hellobike.com/papers/2026-09-18-2609-21449
> 频道：papers | 更新日期：2026-09-18 | 标签：arxiv, cs.RO
> 核心摘要：ME-Dex-1.0将异构触觉与视觉、动作联合建模，基于Mixture-of-Transformers和数据引擎，在仿真与真机操纵中验证。

# ME-Dex 1.0：将异构触觉感知引入世界动作建模

::: v-pre
> 原文链接：[ME-Dex 1.0: Bringing Heterogeneous Tactile Sensing into World Action Modeling](https://arxiv.org/abs/2609.21449)（arXiv 2609.21449 · 发布日期：2026-09-18）

**ME-Dex 1.0: Bringing Heterogeneous Tactile Sensing into World Action Modeling**

ME-Dex-1.0将异构触觉与视觉、动作联合建模，基于Mixture-of-Transformers和数据引擎，在仿真与真机操纵中验证。

## 核心要点

- 现有方法多将触觉作为条件输入，未联合预测未来触觉状态、视觉观测与动作。
- 模型包含Video、Tactile与Action专家，以flow matching训练，并用共享注意力连接中间层。
- Canonical Hand Model与Unified Tactile Autoencoder将不同触觉布局映射到共享空间。
- 在RoboTwin、DexJoCo、ManiFeel仿真及真机评估中，触觉夹爪和灵巧手操纵性能提升。

**验证范围与局限**：验证范围包括三个仿真平台和真实机器人评估，摘要未说明真机数量或具体任务细节。

## 英文摘要 (Abstract)

World Action Models bring the predictive capabilities of video models into robot action generation, providing a rich foundation for modeling future visual states. Tactile sensing complements this foundation with direct measurements of physical interaction. Some existing methods use tactile features as conditioning inputs without jointly predicting future tactile states, visual observations, and actions. Our key insight is that tactile signals, like video, provide observations of the evolving world state and should be modeled as future observations alongside video. We present ME-Dex-1.0 (MachEmbodied-Dex-1.0), a unified World Action Tactile Model for joint visual, tactile, and action learning. ME-Dex-1.0 adopts a Mixture-of-Transformers architecture comprising a Video Expert, a Tactile Expert, and an Action Expert, all trained with flow matching. We use shared attention connects the experts in intermediate layers, allowing action generation to draw on learned representations of visual and tactile dynamics during joint denoising. To support multi-source heterogeneous tactile inputs, a Canonical Hand Model and a Unified Tactile Autoencoder map tactile observations from different embodiments and sensing layouts into shared spatial and latent spaces. To address the limited availability of paired visual, tactile, and action data, we develop the Agentic Tactile Data Engine, an agent-based data production platform. It supplements RoboTwin and DexJoCo with tactile data recorded directly from force sensors during trajectory replay in simulation. Experiments on the RoboTwin, DexJoCo, and ManiFeel simulation platforms, together with real robot evaluations, demonstrate improved manipulation performance using both grippers and dexterous hands equipped with tactile sensing.

作者：Xuancheng Zhang, Xuetao Liu, Qianying Tang, Jizhe Wang, Runsheng Wang, Zhijing Cheng, Bochen Lin, Wen, Haoran, Ming Li, Kun Zhan
:::

## 原始参考来源 (Sources)

- [arXiv 2609.21449：ME-Dex 1.0: Bringing Heterogeneous Tactile Sensing into World Action Modeling](https://arxiv.org/abs/2609.21449)