外观
论文arXiv 2610.1247020 天前
Dex-One2Many：单视频驱动灵巧操作零样本迁移，未见场景成功率超基线71%
本文由 AI 根据公开内容整理，可能有误，请以原文为准：Dex-One2Many: Learning Dexterous Manipulation from a Single Human Demonstration（arXiv 2610.12470）
核心内容
针对单人类视频学习灵巧操作中泛化性差与高维探索难的矛盾，研究提出Dex-One2Many真-仿-真框架。该方法将视频抽象为时序场景图作为生成约束与密集奖励源，指导RL策略在仿真中高效探索。实验覆盖5项工具使用与操作任务，系统在未见过的物体位姿与抓取方式下，性能较基线提升71%，且在已见配置中也高出6.5%，验证了从单一演示到广泛泛化的可行性。
技术上，场景图约束对象间拓扑关系而非绝对位姿，使重置状态采样能覆盖视频外的多样初始条件，配合分阶段密集奖励缩短探索路径。系统完全在仿真环境训练，无需真实机器人示教数据即可零样本部署至实体多指灵巧手。相比纯运动模仿方法，该架构通过结构化先验解耦了动作复现与任务逻辑，有效克服了传统RL在复杂多阶段任务中因缺乏引导导致的样本效率低下问题。
关键看点
- 场景图引导RL：将单视频抽象为时序场景图，提供拓扑约束与分阶段密集奖励，解决高维空间探索难题
- 零样本真机迁移：策略全仿真训练，无需真实示教数据即可直接部署至多指灵巧手完成5类操作任务
- 强泛化性能验证：在未见物体位姿与抓取配置下成功率超基线71%，已见场景亦提升6.5%
- 生成式状态重置：基于关系约束采样多样化重置状态，使训练分布自动扩展至视频演示范围之外
工程与产业观察
该工作证明了结构化语义表征（场景图）可作为连接单模态人类视频与机器人策略的高效桥梁，大幅降低灵巧操作对海量真机数据的依赖。对于具身智能研发而言，这种“视频即约束”的范式为低成本构建泛化操作技能提供了可复现的技术路径，尤其适用于长序列、多阶段复杂任务的冷启动训练。
英文摘要 (Abstract)
While learning dexterous manipulation from a single human video offers a promising alternative to costly robot demonstrations, many recent methods predominantly imitate demonstrated motions. Such strict motion matching often limits generalization to initial object poses, goal poses, and grasps not shown in the video. Alternatively, discovering a policy via reinforcement learning (RL) allows for broad generalization, but without prior guidance, it struggles with high-dimensional exploration in complex, multi-stage tasks. To address these coupled generalization and exploration challenges, we present Dex-One2Many, a real-to-sim-to-real framework that learns a generalizable dexterous manipulation policy from a single human video. Our key insight is to abstract the video into sequential scene graphs that guide RL, enabling efficient exploration while preserving broad generalizability. The graphs serve as generative constraints for sampling diverse reset states and provide dense rewards for each stage. Because the graphs constrain relations rather than exact poses, these reset states cover object poses and grasps beyond the video, while initializing each stage from them with dense rewards keeps exploration short and guided. Trained entirely in simulation, Dex-One2Many transfers zero-shot to a real multi-fingered hand. Across five tool-use and manipulation tasks, Dex-One2Many exceeds baselines by 6.5% in seen configurations, while its robust generalization widens this gap to 71% in unseen scenarios.
作者：Jusuk Lee, Sungha Kim, Yeonsoo Park, Jonguk Cheon, Yoonkyo Jung, Yongjun You, H. Jin Kim, Jia-Bin Huang, Furong Huang, Youngseok Jang, Seungjae Lee
来源
本页事实对应的公开页面，以原文为准。