外观
论文arXiv 2610.1576917 天前
Hand2Robot：面向机器人的基于人类示范的物体中心4D手部重建
原文链接：Hand2Robot: Object-Centric 4D Hand Reconstruction from Human Demonstrations for Robotics（arXiv 2610.15769 · 发布日期：2026-10-11）
Hand2Robot: Object-Centric 4D Hand Reconstruction from Human Demonstrations for Robotics
Hand2Robot从人类任务视频中以物体为中心重建4D手部，并经交互优化和重定向生成监督，在基准与真机任务上验证。
核心要点
- 人类示范能捕获操作，但手部运动不能直接作为机器人动作监督。
- 该框架提取交互片段、重建物体几何，并生成机器人动作监督。
- 物体跟踪结合观测置信度与时间运动约束，选择全局一致姿态轨迹。
- 在DexYCB和HO3D-v3上提升物体姿态准确性与手-物交互质量。
验证范围与局限：验证范围包括DexYCB、HO3D-v3和三个真机灵巧操作任务。
英文摘要 (Abstract)
Human demonstrations can capture task-relevant manipulation without using the target robot, but the recorded hand motions cannot directly serve as robot action supervision. Hand reconstruction and motion retargeting provide a direct route to such supervision, yet recovering hand motion alone does not ensure physically consistent hand-object interactions. To address this issue, we introduce Hand2Robot, an object-centric 4D hand reconstruction framework for robot policy post-training. Given videos of humans performing the target task, Hand2Robot extracts interaction clips, reconstructs object geometry, and generates robot action supervision through hand-object interaction optimization and motion retargeting. For object tracking, we combine observation confidence with temporal motion constraints to select a globally consistent object pose trajectory from per-frame pose candidates. For hand refinement, we use object geometry and motion as a reference and combine staged 2D alignment with object-local contact constraints to refine hand trajectories and finger joint configurations. Experiments on DexYCB and HO3D-v3 show improved object pose accuracy and hand-object interaction quality. Experiments on three real-robot dexterous manipulation tasks further show that reconstructed and retargeted human demonstrations can effectively supplement limited robot data for target-task policy post-training.
作者：Zetao Du, Weiqing Wang, Hanbing Zhang, Yimiao Zhou, Ye Shi, Liang Wang, Jingya Wang
来源
本页事实对应的公开页面，以原文为准。