论文/arXiv 2610.14691·18 天前

DexCast：从人类视频中学习真实世界灵巧控制

DEXCAST通过联合HOI重建从单目RGB-D人类视频学习灵巧机械手控制，并在真实世界多样灵巧操作任务中验证。

AI 整理

• 本体 SDK 与控制
• 数据与采集
• 具身模型

原文链接：DexCast: Learning Real-World Dexterous Control from Human Videos（arXiv 2610.14691 · 发布日期：2026-10-10）

DexCast: Learning Real-World Dexterous Control from Human Videos

DEXCAST通过联合HOI重建从单目RGB-D人类视频学习灵巧机械手控制，并在真实世界多样灵巧操作任务中验证。

核心要点

• 现有灵巧模仿学习常依赖昂贵的人工遥操作机器人示范，需大量人力投入。
• DEXCAST直接利用单目RGB-D视频和联合HOI重建学习灵巧机械手控制，无需遥操作。
• 提出DexHand Motion Perception Module，以紧凑潜表示捕获关节协同和时间依赖并条件化扩散策略。
• 大量真实世界实验表明，DEXCAST在多样灵巧操作任务上取得优异表现。

验证范围与局限：验证范围为摘要所述的真实世界多样灵巧操作任务。

英文摘要 (Abstract)

Recent advances in 3D hand-object interaction (HOI) perception have opened new opportunities for learning dexterous robotic skills from human videos. However, existing dexterous imitation learning methods often rely on costly and manually teleoperated robot demonstrations, requiring substantial human effort. Therefore, we build DEXCAST, a framework that learns dexterous robotic hand control directly from monocular RGB-D videos by leveraging joint HOI reconstructions, without requiring any teleoperation. Furthermore, how to model high-dimensional robotic hand motion and extract temporal context to inform future action generation remains a key challenge. To this end, we propose a DexHand Motion Perception Module (DMPM) to capture inter-joint coordination and temporal dependencies in compact latent representations that condition our diffusion policy. Extensive real-world experiments demonstrate that DEXCAST achieves exceptional performance across diverse dexterous manipulation tasks.

作者：Zihan Gao, Yuhong Zhang, Mengzhe Wang, Kaisheng Liu, Shengpeng Li, Siyao Xiao, Runqing Cheng, Xiao Lin, Zhifang Liu, Haoqian Wang

来源

本页事实对应的公开页面，以原文为准。

1. arXiv 2610.14691：DexCast: Learning Real-World Dexterous Control from Human Videos访问 2026-10-21