论文/arXiv 2608.16572·2 个月前

ViHaTeleop：面向灵巧操作学习的低成本轻量级视觉-触觉遥操作系统

ViHaTeleop以视觉追踪和逐指振动反馈采集接触示教，并在仿真与真机接触任务中验证。

AI 整理

• 硬件零部件
• 感知与导航
• 仿真与训练

Franka FR3

原文链接：ViHaTeleop: A Low-Cost, Lightweight Visual-Haptic Teleoperation System for Dexterous Manipulation Learning（arXiv 2608.16572 · 发布日期：2026-08-17）

ViHaTeleop: A Low-Cost, Lightweight Visual-Haptic Teleoperation System for Dexterous Manipulation Learning

ViHaTeleop以视觉追踪和逐指振动反馈采集接触示教，并在仿真与真机接触任务中验证。

核心要点

• 低成本遥操作硬件难以采集高质量、接触关键的灵巧操作示教。
• 系统结合SLAM腕部追踪、相机手部追踪和逐指LRA振动反馈。
• 九名参与者六项任务中，触觉使全部任务成功率提升2.2至15.6个百分点。
• 初步策略训练显示，触觉线索在插孔子任务较纯视觉提升17个百分点。

验证范围与局限：验证包括Franka + LEAP Hand + 9DTact在真实与仿真环境中的六项接触关键任务，以及初步下游策略训练。

英文摘要 (Abstract)

Learning from demonstration is a promising approach for dexterous manipulation, but collecting high-quality contact-critical demonstrations remains difficult with low-cost teleoperation hardware. We present ViHaTeleop, a lightweight (0.7 kg), low-cost ($550) visual-haptic teleoperation system with SLAM-based wrist tracking, camera-based hand tracking, and finger-wise vibrotactile feedback through Linear Resonant Actuators (LRA). The system includes several design choices (LED illumination, fisheye hand camera, and tactile-aware retargeting constraints) and is deployed on Franka + LEAP Hand + 9DTact in both real and simulated environments. Under matched with/without-haptic conditions with nine participants across six contact-critical tasks, haptics improved success rates across all tasks (+2.2 to +15.6 percentage points), while completion-time effects were task-dependent. Subjective ratings showed significant gains in contact clarity and grasp confidence in both simulation and real-world settings (Wilcoxon signed-rank, $p<0.05$). We also integrate a lightweight depth-camera-based tactile proxy in Isaac Sim, enabling a full pipeline from multi-modal demonstration collection to visual-tactile policy training. Preliminary downstream validation by training visual-tactile policies from collected demonstrations shows tactile cues benefit contact-critical subtasks (peg-in-hole: +17 percentage points over vision-only).

作者：Fucai Zhu, Yanhou Lai, Paul Maestre, Koichi Hashimoto

来源

本页事实对应的公开页面，以原文为准。

1. arXiv 2608.16572：ViHaTeleop: A Low-Cost, Lightweight Visual-Haptic Teleoperation System for Dexterous Manipulation Learning访问 2026-10-21