外观
论文arXiv 2610.2179213 天前
面向可泛化操作的单次人类视频轨迹扩散
原文链接：Trajectory-based Diffusion from One-shot Human Video for Generalizable Manipulation（arXiv 2610.21792 · 发布日期：2026-10-15）
Trajectory-based Diffusion from One-shot Human Video for Generalizable Manipulation
TDV从单次人类视频提取物体轨迹并生成数据，缓解视觉与具身差异，在RLBench仿真和真实实验验证可泛化操作。
核心要点
- 现有模仿学习依赖额外遥操作系统采集数据，扩展性受限；人类视频又存在视觉域差异且缺少动作标签。
- TDV利用即插即用6D位姿估计，从人类视频中提取任务相关物体轨迹，降低无关背景和视角变化影响。
- 轨迹作为模态无关中间表示，独立于视觉域和不同具身形式，可支持高效的数据生成。
- 方法在RLBench七个仿真任务上优于既有方法并接近100%成功率，真实实验显示可由单次人类示范实现跨具身泛化。
验证范围与局限：验证范围包括RLBench仿真七个任务和真实世界实验，摘要未说明具体机器人数量与真实任务细节。
英文摘要 (Abstract)
Imitation learning enables robots to perform complex tasks by leveraging human-collected demonstrations. However, most existing approaches require labor-intensive data collection with additional teleoperation systems, which hinders scalability. Human videos represent a natural source of data that implicitly contain knowledge of manipulation behaviors. However, transferring them to robots remains challenging due to the visual domain gap and the absence of explicit action labels. In this work, we introduce TDV, a trajectory-based framework that leverages data generation to enable generalizable manipulation. TDV focuses on task-relevant objects by leveraging plug-and-play 6D pose estimation techniques to extract object trajectories from human videos, thereby mitigating the impact of irrelevant backgrounds and varying viewpoints. As a modality-agnostic intermediate representation, trajectories are inherently independent of the visual domains and different embodiments, enabling efficient data generation. Our method outperforms previous approaches on RLBench simulation tasks, achieving near 100% success rates across seven tasks. In real-world experiments, TDV enables cross-embodiment generalization manipulation from one-shot human demonstrations by leveraging efficient trajectory generation techniques.
作者：Guoping Pan, Xinmiao Yu, Wannan Cao, Jincheng Li, Baolin Jiang, Yizhe Li, Yi Cheng, Houde Liu
来源
本页事实对应的公开页面，以原文为准。