外观
论文arXiv 2608.020692 个月前
Open-DiffLoco：用于可部署盲四足运动的开源可微分学习
Open-DiffLoco用MJX中的SHAC和可微分仿真训练本体感知盲四足策略，并在Unitree Go2真机上验证。
宇树 Go2原文链接：Open-DiffLoco: Open-Source Differentiable Learning for Deployable Blind Quadruped Locomotion（arXiv 2608.02069 · 发布日期：2026-08-03）
Open-DiffLoco: Open-Source Differentiable Learning for Deployable Blind Quadruped Locomotion
Open-DiffLoco用MJX中的SHAC和可微分仿真训练本体感知盲四足策略，并在Unitree Go2真机上验证。
核心要点
- 传统强化学习常需复杂奖励设计和高训练成本，端到端迁移开源工具仍有限。
- 框架在MuJoCo XLA中实现SHAC，去除特权观测和参考轨迹，采用简化奖励。
- 真机策略可跟踪全向速度命令，误差低于0.2 m/s，速度超过1 m/s，并能应对不平地形和侧向推搡。
- 提出JAVE监督评论器Jacobian以改进早期一阶策略梯度训练，代码和部署视频已公开。
验证范围与局限：验证范围为仿真训练后部署到一台Unitree Go2四足机器人，测试速度跟踪、不平地形和外部扰动。
英文摘要 (Abstract)
Developing deployable locomotion policies through conventional reinforcement learning often requires complex reward engineering and expensive training times. While differentiable simulation offers a highly efficient alternative, open-source tools capable of end-to-end transfer of these policies to physical hardware remain limited. This paper introduces Open-DiffLoco, an open-source framework for training deployable blind quadruped locomotion policies with differentiable simulation. The framework implements the Short-Horizon Actor-Critic (SHAC) algorithm in MuJoCo XLA (MJX) and trains a proprioceptive policy that transfers to real-world hardware. The deployed policy removes privileged actor observations, including base linear velocity, and does not rely on reference trajectories. It also uses a substantially simplified reward function, enabling the robot to discover walking patterns without the complex auxiliary rewards typically used in conventional reinforcement learning pipelines. When deployed on physical hardware (a Unitree Go2 quadruped), the trained policy tracks omnidirectional velocity commands with root-mean-square error below 0.2 m/s, reaches speeds above 1 m/s, and remains robust to uneven terrain and external physical disturbances, such as lateral pushes. Across the reported configurations, training uses under 6 GB of VRAM on a single NVIDIA GeForce RTX 5080 GPU and completes in approximately 20-60 minutes. As an algorithmic extension to SHAC, we propose Jacobian-Augmented Value Estimation (JAVE), which supervises the critic Jacobians to improve early first-order policy-gradient training. To our knowledge, Open-DiffLoco is the first open-source framework for training deployable locomotion policies using differentiable simulation. Deployment videos and source code are available at: https://diffloco.martin-opat.com/
作者：Martin Opat
来源
本页事实对应的公开页面，以原文为准。