论文arXiv 2611.163379 天前
混合地形建模实现百倍并行强化学习,加速机器人坍塌撤退策略收敛
本文由 AI 根据公开内容整理,可能有误,请以原文为准:Memory-Efficient Parallel Policy Learning for Collapse-Reactive Retreat via Hybrid Terrain Modeling(arXiv 2611.16337)
核心内容
针对灾害现场不稳定斜坡中机器人因支撑丧失需紧急撤退的场景,传统大规模并行强化学习受限于刚体坍塌仿真的内存开销。研究提出一种内存高效的并行策略学习框架,核心在于利用混合地形建模替代全刚体仿真,解决共享可坍塌地形导致的间接干扰问题。该方案在相同GPU显存限制下,支持比精细刚体块地形多100倍以上的并行环境,使策略收敛速度提升超50倍,且在Sim-to-sim及真机实验中均验证了多种坍塌条件下的有效撤退能力。
技术架构采用混合地形建模,将不可坍塌区域保留刚体仿真,可坍塌区域则用虚拟接触力替代。核心组件State-Multiplexed Contact Model共享地形几何与接触响应定律,同时维护环境特定的坍塌状态,从而无需为每个并行环境复制可坍塌地形对象即可获取独立经验。该框架训练本体感知策略以兼顾正常运动与坍塌反应,显著提升了样本效率。相比计算昂贵的全刚体坍塌建模,该方法在保证撤退性能相当的前提下,大幅降低了显存占用并加速了训练进程。
关键看点
- 混合地形建模:结合不可坍塌区刚体仿真与可坍塌区虚拟接触力,消除共享地形间接干扰,避免复制地形对象带来的显存爆炸
- 状态复用接触模型:共享几何与接触响应定律并维护独立坍塌状态,同等显存下并行环境数量较精细刚体地形提升超100倍
- 训练效率跃升:凭借高可扩展性与样本效率,策略收敛速度加快50倍以上,且保持与全刚体仿真相当的撤退性能
- 本体感知策略:单一策略网络同时处理正常运动与坍塌反应,经Sim-to-sim及真机实验验证在多种坍塌条件下有效
工程与产业观察
该工作为非结构化地形具身智能训练提供了关键的工程降本范式。通过解耦物理几何与接触响应,突破了复杂动态环境RL训练的显存墙,使百级并行成为常态。这对四足/人形机器人在野外、废墟等高风险场景的Sim-to-Real落地具有直接参考价值,证明了在非完全保真仿真下仍可高效习得高鲁棒性安全策略。
英文摘要 (Abstract)
At disaster sites and on unstable slopes, robot contact can collapse uninspected terrain, requiring retreat upon support loss. Massively parallel reinforcement learning of collapse-reactive retreat requires independent state--action experiences. However, on shared rigid-body collapsible terrain, one robot's collapse alters others' contacts without direct interaction. We call this indirect interference. Duplicating terrain objects prevents this but is memory-intensive, limiting parallel sampling. Our key insight is that policy learning does not necessarily require rigid-body collapse modeling, which is feasible but computationally expensive; collapse-induced contact responses can suffice. We therefore propose a memory-efficient parallel policy-learning framework for collapse-reactive retreat via hybrid terrain modeling. Hybrid terrain modeling combines rigid-body simulation of non-collapsible terrain with virtual contact forces for collapsible regions. A State-Multiplexed Contact Model shares terrain geometry and a contact-response law while maintaining environment-specific collapse states, enabling independent experiences without terrain objects for collapsible regions. The framework trains a proprioceptive policy for both normal locomotion and collapse-reactive retreat. It supports over 100 times as many parallel environments as fine-resolution rigid-block terrain within the same GPU memory limit. This scalability and improved sample efficiency enable over 50 times faster policy convergence with comparable retreat performance. Sim-to-sim and real-robot experiments demonstrate retreat under multiple collapse conditions. A supplementary video is available at: https://youtu.be/_RrhwjzEPE0
作者:Yuki Kadokawa, Ritsuki Matsunaga, Yuqi Yang, Edgar Anarossi, Takamitsu Matsubara
外部来源
本页事实对应的公开页面,以原文为准。(点击将离开本站)