外观
论文arXiv 2609.275471 个月前
EBRL：通过多粒度资源管理实现异步具身 RL
原文链接：EBRL: Asynchronous Embodied RL by Multi-Grained Resource Management（arXiv 2609.27547 · 发布日期：2026-09-23）
EBRL: Asynchronous Embodied RL by Multi-Grained Resource Management
EBRL面向具身RL训练资源浪费问题，以异步流水调度和细粒度资源管理提升仿真基准上的训练效率。
核心要点
- 具身RL包含环境仿真、动作生成和模型更新，各阶段CPU/GPU需求异构，资源利用困难。
- EBRL通过异步流水调度重叠rollout与训练，并让环境组间仿真和生成流水化、环境独立执行。
- 细粒度资源管理器池化CPU核心和GPU流式多处理器，依据阶段画像与运行反馈调整配额和批大小。
- 在RLinf上实现，并以四种具身策略和四个仿真基准评估，rollout吞吐达1.30-3.47倍，训练收敛达2.5倍。
验证范围与局限：验证范围为异构GPU测试平台上的四种具身策略和四个仿真基准。
英文摘要 (Abstract)
Embodied reinforcement learning (RL) improves model capabilities with a pipeline of environment simulation, action generation, and model updates. These stages show heterogeneous CPU and GPU demands, making efficient resource utilization difficult. Recent systems overlap rollout (simulation and generation) with training for efficiency, but exclusive GPU allocation and synchronized barrier in rollout still leave substantial hardware resource waste. In this paper, we present EBRL, an asynchronous embodied RL training system with two core techniques. The asynchronous pipelined scheduler overlaps rollout and training, pipelines simulation and generation across environment groups, and carries out each environment independently, eliminating synchronization stalls. The fine-grained resource manager pools CPU cores and GPU streaming multiprocessors, and uses stage profiles and runtime feedback to adjust resource quotas and batch sizes to meet the shifting demands among stages. We implement EBRL on RLinf and evaluate it with four embodied policies and four simulation benchmarks across heterogeneous GPU testbeds. Experiments show that EBRL achieves 1.30-3.47 times the end-to-end rollout throughput and 2.5 times of training convergency compared to the SOTA embodied RL systems.
作者：Liang Mi, Weijun Wang, Bowen Gao, Tianze Yu, Zixu Hao, Han Xiao, Xin Ding, Mingzhe Huang, Xin He, Lu Shi
来源
本页事实对应的公开页面，以原文为准。