# 现在你感觉到了，现在你看见我：基于数字孪生的灵巧操作遥操作界面

> 原文链接：https://robotics.hellobike.com/papers/2026-10-04-2610-05081
> 频道：papers | 更新日期：2026-10-04 | 标签：arxiv, cs.RO
> 核心摘要：针对裸手跟踪遥操作缺少力反馈和遮挡视觉反馈的问题，提出数字孪生AR界面，并在两项用户研究任务中评估。

# 现在你感觉到了，现在你看见我：基于数字孪生的灵巧操作遥操作界面

::: v-pre
> 原文链接：[Now You Feel It, Now You See Me: Digital-Twin-based Teleoperation Interface for Dexterous Manipulation](https://arxiv.org/abs/2610.05081)（arXiv 2610.05081 · 发布日期：2026-10-04）

**Now You Feel It, Now You See Me: Digital-Twin-based Teleoperation Interface for Dexterous Manipulation**

针对裸手跟踪遥操作缺少力反馈和遮挡视觉反馈的问题，提出数字孪生AR界面，并在两项用户研究任务中评估。

## 核心要点

- 裸手跟踪便于控制机器人手并示教协调手指动作，但缺少力反馈和遮挡区域视觉反馈。
- 方法集成裸手跟踪、双手机器人手以及机器人和任务物体的虚拟表示，构建数字孪生AR遥操作界面。
- 界面在机器人手网格上渲染触觉测量作为视觉-力反馈，并提供虚拟视图面板或遮挡感知透明渲染。
- 用户研究考察力可视化和视觉辅助如何支持需要精细力调节及遮挡下操作的示教采集。

**验证范围与局限**：验证范围为涉及Task 1和Task 2的用户研究，任务需要精细力调节和遮挡下操作。

## 英文摘要 (Abstract)

Teleoperation is becoming increasingly important for collecting high-quality demonstrations to teach robots dexterous manipulation skills. For dexterous manipulation, bare-hand tracking provides a practical way to control robotic hands and demonstrate coordinated finger movements without gloves or exoskeletons. However, this type of teleoperation faces two key feedback limitations: a lack of force feedback and visual feedback of occluded region. The absence of force feedback hinders precise and safe manipulation, as operators must infer contact force visually rather than feel them directly. Occlusion by objects or other robot parts impairs the assessment of hand positions, approach distances, and pre-grasp configurations suited to the object's shape. To address these limitations, we propose a digital-twin-based augmented reality (AR) teleoperation interface that integrates bare-hand tracking, bimanual robotic hands, and virtual representations of the robot and task objects. The interface renders tactile measurements on robot hand meshes as visuo-force feedback. Furthermore, it offers a user-controlled virtual view panel or occlusion-aware transparency rendering to provide occlusion-mitigating visual feedback. We evaluated the interface through user studies involving Task 1 and Task 2, examining how force visualization and visual assistance support demonstration collection in tasks requiring careful force regulation and manipulation under occlusion.

作者：심영찬, Kyutae Lee, JooYun Kim, Jaeseong Hwang, Harim Ji, Yongseok Lee
:::

## 原始参考来源 (Sources)

- [arXiv 2610.05081：Now You Feel It, Now You See Me: Digital-Twin-based Teleoperation Interface for Dexterous Manipulation](https://arxiv.org/abs/2610.05081)