# SGS算法突破百万级并行RL探索瓶颈，实现四足与装配零样本迁移

> 原文链接：https://robotics.hellobike.com/papers/2026-10-08-2610-12465
> 频道：papers | 更新日期：2026-10-08 | 标签：auto, arxiv
> 核心摘要：针对大规模强化学习中均匀采样导致的算力浪费问题，提出Success Guided Sampling自适应采样器。在超百万并行环境中验证，成功解决复杂地形四足运动与接触丰富装配任务，并实现RGB策略真机零样本迁移。

# SGS算法突破百万级并行RL探索瓶颈，实现四足与装配零样本迁移

::: v-pre
> 本文由 AI 根据公开内容整理，可能有误，请以原文为准：[A Balanced Data Diet: Addressing the Exploration Bottleneck in Mega-Scale RL for Robot Control](https://arxiv.org/abs/2610.12465)（arXiv 2610.12465）

## 核心内容

针对通用机器人Sim-to-RL训练中均匀采样导致学习信号浪费的痛点，研究提出Success Guided Sampling（SGS）自适应采样方法。该方法将训练重心动态聚焦于策略能力边界，在高达$2^{20}$（超100万）并行环境的Mega-Scale RL实验中，成功解决了传统方法无法攻克的多地形四足运动及接触丰富的精密装配任务，显著提升了大规模并行仿真的数据利用效率。

SGS通过识别策略掌握程度，避免在已 mastered 或不可行配置上浪费算力，有效缓解了动态精准任务的探索瓶颈。实验表明该机制使RL训练能充分利用批量经验，并将学到的操作策略蒸馏为基于RGB输入的视觉策略。最终在真实硬件上实现了多个高难度装配任务的零样本迁移，验证了从纯仿真到物理世界的泛化能力，项目代码与文档已在sgs-rl.github.io开源。

## 关键看点

- **SGS自适应采样**：动态聚焦策略能力边界配置，消除百万级并行RL中均匀采样造成的无效学习信号浪费
- **Mega-Scale验证**：在$2^{20}$并行环境规模下，攻克多地形四足运动与接触丰富装配等先前失败任务
- **真机零样本迁移**：将仿真RL策略蒸馏为RGB视觉策略，无需微调即可在真实硬件完成复杂装配
- **去工程化先验**：摆脱对Shaped Rewards和演示数据的依赖，仅靠多样化重置与自适应采样驱动学习

## 工程与产业观察

SGS证明了在Mega-Scale RL中，智能采样比单纯堆砌并行环境数量更关键。对于具身智能研发，该方案提供了降低Sim-to-Real对齐成本、提升数据利用率的具体路径，特别是其RGB策略蒸馏与零样本迁移能力，为缺乏大量真机数据的团队提供了可复现的高性能基线。

## 英文摘要 (Abstract)

General-purpose robots must perform a wide range of tasks from agile locomotion to dexterous manipulation. While sim-to-real reinforcement learning (RL) has proven to be a useful tool for this goal, current RL pipelines depend on engineering-heavy, per-task structural priors such as shaped rewards and demonstrations. Recent work has shown that diverse simulator resets, combined with massively parallel simulation, can alleviate much of this engineering burden on several manipulation problems. However, we find that naively scaling this paradigm to more precise or dynamic problems remains non-trivial. While simulator resets can help with exploration, uniformly sampling over this distribution wastes a growing fraction of learning experience on task configurations the policy has already mastered or cannot yet attempt. This makes it challenging to see the expected benefits of scaling parallel environments for RL, since much of the learning signal in a batch is wasted during learning. To mitigate this, we introduce Success Guided Sampling (SGS), a simple adaptive sampler that concentrates RL training on task configurations around the frontier of the policy's capabilities. Doing so allows large-scale simulated RL to make the most out of the experience in a batch, enabling much more effective scaling to large-scale parallel simulation. Across experiments using up to $2^{20}$ (over one million) parallel environments, SGS enables RL to solve challenging multi-terrain quadruped locomotion and contact-rich assembly tasks that prior methods fail to solve. Finally, we distill the learned manipulation policies into RGB-based policies and demonstrate zero-shot transfer to several challenging assembly tasks on real hardware. Project website: https://sgs-rl.github.io/.

作者：Octi Zhang, Mateo Guaman Castro, Patrick Yin, Ignacio Dagnino, Abhishek Gupta, Rosario Scalise, Byron Boots
:::

## 原始参考来源 (Sources)

- [arXiv 2610.12465：A Balanced Data Diet: Addressing the Exploration Bottleneck in Mega-Scale RL for Robot Control](https://arxiv.org/abs/2610.12465)