论文arXiv 2611.235023 天前
TAG平台验证物理耦合系统下强化学习策略在线自适应恢复能力
本文由 AI 根据公开内容整理,可能有误,请以原文为准:Learning and Control in Physically Coupled Dynamical Systems(arXiv 2611.23502)
核心内容
arXiv论文提出TAG(Testbed for Autonomous Games)紧凑低成本机器人平台,专用于研究物理耦合动力系统中的学习与控制问题。该平台核心挑战在于解决子系统间动态交互对独立策略学习的干扰。研究采用互补强化学习范式,分别利用基于模型和无模型方法训练迷宫与倒立摆控制策略。关键发现是:当倒立摆引入物理耦合导致迷宫策略成功率瞬间大幅下降时,系统能通过在线适应机制,在极少学习回合内将性能恢复至耦合前水平,证明独立训练策略在强耦合环境下仍具备鲁棒性。
TAG硬件由主动倾斜转向的球体迷宫板与顶部安装的旋转倒立摆组成,二者通过反应扭矩与振动形成双向物理扰动:倒立摆运动干扰迷宫板姿态,而迷宫求解策略驱动的板体倾斜同时破坏倒立摆平衡。针对此耦合特性,研究设计了差异化算法架构:迷宫控制采用Model-based RL以应对复杂接触动力学,倒立摆控制采用Model-free RL处理高频非线性响应。实验数据表明,尽管两策略独立训练且存在显著机械耦合,系统仍能实现双任务稳定协同控制,为具身智能在多自由度耦合场景下的Sim-to-Real迁移提供了验证基准。
关键看点
- 物理耦合机制:TAG平台通过倒立摆反应扭矩与迷宫板倾斜构建双向扰动链路,量化验证强耦合下策略性能的瞬时衰减与在线恢复过程
- 互补RL范式:迷宫子系统采用Model-based RL处理接触动力学,倒立摆采用Model-free RL应对非线性,独立训练策略在耦合态下实现协同稳定
- 在线适应能力:引入倒立摆致迷宫策略成功率骤降,但系统仅需少量learning episodes即可在线适应并完全恢复至解耦前基准性能
工程与产业观察
TAG平台揭示了具身智能在物理耦合环境中的关键工程规律:独立训练的策略虽会因未建模动力学遭遇性能断崖,但具备快速在线修正潜力。这提示开发者在设计多执行器协同系统时,应优先保障策略的在线适应带宽而非过度追求离线完美模型,互补RL架构可作为处理异构动力学耦合的有效范式参考。
英文摘要 (Abstract)
We introduce TAG (Testbed for Autonomous Games), a compact, low-cost robotic platform for studying learning and control in physically coupled dynamical systems. TAG combines an actively tilted and steered ball-in-maze board with a top-mounted rotary inverted pendulum. The pendulum's reaction torque and vibration disturb the board and marble, while the board's tilt, driven by the maze-solving policy, simultaneously disturbs the pendulum's balance. The main challenge in learning control policies for each subsystem is accounting for the impact of one subsystem's dynamics on the other's learning process. To address this challenge, we present complementary reinforcement learning paradigms, leveraging model-based and model-free approaches to learn the control policies for the maze and pendulum, respectively. Our key finding is that introducing the pendulum causes an immediate, sharp drop in the maze policy's success rate, but the policy adapts online and recovers to its pre-coupling performance within a small number of learning episodes. Experimental results show that both policies, trained independently, successfully control the maze board and the pendulum despite their physical coupling.
作者:Bao Trung Truong, Dat Thanh Nguyen, Thinh T. Doan
外部来源
本页事实对应的公开页面,以原文为准。(点击将离开本站)