外观
论文HF Daily Papers · arXiv 2610.320935 天前
DiscoDemo：RL生成多样化演示数据提升模仿学习真机成功率至78%
本文由 AI 根据公开内容整理，可能有误，请以原文为准：DiscoDemo: Discovering Efficient and Diverse Robot Demonstrations for Imitation Learning（HF Daily Papers · arXiv 2610.32093）
核心内容
针对行为克隆对训练数据覆盖率敏感且采集成本高的问题，研究团队提出DiscoDemo框架，将强化学习重新定位为下游视觉运动模仿学习的数据生成器而非部署控制器。核心假设是多种成功解模式比单一窄解能覆盖更广的成功轨迹空间。实验显示，在四项操作任务中，该方法使下游策略在仿真中的平均成功率从最强基线的40%提升至62%，真机环境从39%跃升至78%，证明了解多样性对预训练流匹配VLA模型的有效性。
技术上，DiscoDemo训练了一个特权潜在条件RL生成器，采用并行反向课程学习与成功门控多样性目标，在确保任务成功率的同时发现多样化解决方案。生成器产出的多条成功闭环行为经安全过滤后导出为静态数据集供模仿学习使用。结果表明，这种基于RL的多样化数据生成与纠错数据采集具有互补性，能有效解决单一演示导致的分布外泛化差问题，为具身智能数据工程提供了新的自动化合成范式。
关键看点
- 真机性能突破：在真实机器人操作测试中，下游视觉运动策略平均成功率由39%显著提升至78%，验证了RL生成数据在实际物理环境中的有效性。
- 仿真基准提升：四项操作任务仿真测试中，下游策略成功率从最强基线的40%提高至62%，证明多模态成功解覆盖了更广泛的状态空间。
- 算法核心机制：采用特权潜在条件RL生成器，结合并行反向课程与成功门控多样性目标，在保证任务成功前提下自动发现并导出多样化闭环轨迹。
- 数据工程范式：将RL从部署控制器转变为数据生成工具，生成的静态数据集与纠错采集互补，有效增强了预训练流匹配VLA模型的泛化能力。
工程与产业观察
该工作确立了“RL即数据引擎”的工程范式，解决了具身模仿学习中高质量多模态数据稀缺痛点。对于算法工程师，这意味着可通过自动化生成多样化成功轨迹来弥补人工示教不足；对于产业决策者，此方法降低了真机数据采集边际成本，加速了通用操作策略从仿真到真机的迁移验证效率，是构建规模化具身数据集的关键技术路径。
英文摘要 (Abstract)
Behavior cloning for robotic manipulation is highly sensitive to the coverage of its training data, yet collecting diverse successful demonstrations remains costly. We study reinforcement learning (RL) not as a deployable controller, but as a data generator for downstream visuomotor imitation learning. Our key premise is that multiple successful solution modes cover a broader range of successful trajectories than repeatedly collecting a single narrow solution. We introduce DiscoDemo, which trains a privileged latent-conditioned RL generator with a parallel reverse curriculum and a success-gated diversity objective to discover diverse solutions while preserving task success. The trained generator produces multiple successful closed-loop behaviors, which are safety-filtered and exported as a static dataset for downstream imitation learning. Across four manipulation tasks, DiscoDemo produces broader successful-state coverage and improves downstream visuomotor policies in both simulation and the real world. Averaged across tasks, downstream success increases to 62% from 40% for the strongest baseline in simulation, and to 78% from 39% on the real robot, respectively. These results suggest that successful solution diversity improves imitation learning for a pretrained flow-matching VLA and remains complementary to corrective data collection. Project page: https://davian-robotics.github.io/DiscoDemo
Hugging Face Daily Papers 热度：19 upvotes 作者：Minho Park, Kinam Kim, Donghu Kim, Byungkun Lee, Dongyoon Hwang, Yongjae Shin, Junha Hyung, Hojoon Lee, Jaegul Choo
来源
本页事实对应的公开页面，以原文为准。