# 强化学习实现柔性线状物操作零样本迁移，MuJoCo并行训练提速

> 原文链接：https://robotics.hellobike.com/papers/2026-11-06-2611-07261-36236881
> 频道：papers | 更新日期：2026-11-06 | 标签：auto, arxiv
> 核心摘要：研究证实GPU并行MuJoCo在柔性线状物RL训练中收敛优于CPU，结合相对关键点观测与非对称Actor-Critic架构，仿真单点跟踪成功率达96.4%，并通过无标记RGB-D估计实现零样本Sim-to-Real迁移，物理实验成功率超80%。

# 强化学习实现柔性线状物操作零样本迁移，MuJoCo并行训练提速

::: v-pre
> 本文由 AI 根据公开内容整理，可能有误，请以原文为准：[An Empirical Study of Reinforcement Learning for Keypoint Positioning of Deformable Linear Objects](https://arxiv.org/abs/2611.07261)（arXiv 2611.07261）

## 核心内容

针对柔性线状物（DLO）高维非线性操控难题，该研究系统评估了强化学习在平面操作中的训练效率与零样本迁移能力。对比三种PPO训练栈发现，GPU并行MuJoCo Playground在相同时间预算下收敛速度与性能一致性显著优于CPU方案。引入相对关键点观测与非对称Actor-Critic后，策略在含噪声、延迟及域随机化的仿真环境中，单点跟踪、扰动恢复及两点跟踪成功率分别达96.4%、98.0%和88.8%。

在Sim-to-Real环节，系统采用无标记RGB-D关键点估计替代真实世界微调，验证了策略的泛化鲁棒性。物理实验中，单点跟踪、扰动恢复及两点跟踪任务分别取得9/10、10/10和8/10的成功率。该工作证明了基于视觉反馈的RL策略配合高效仿真器，可在不依赖真机数据的情况下解决DLO接触动力学建模难与数据采集成本高的问题，为柔性物体自动化操作提供了可复现的工程范式。

## 关键看点

- **训练加速**：GPU并行MuJoCo Playground栈在同等墙钟时间内，收敛速度与性能稳定性显著优于传统CPU训练方案
- **算法优化**：采用相对关键点观测与非对称Actor-Critic架构，有效提升柔性体操作策略的学习效率与鲁棒性
- **仿真指标**：在注入噪声、延迟及域随机化条件下，单点跟踪成功率96.4%，扰动恢复达98.0%
- **零样本迁移**：基于无标记RGB-D关键点估计，无需真机微调即实现物理验证，两点跟踪实测成功率80%

## 工程与产业观察

该研究确立了GPU加速仿真与相对状态表征在柔性体RL中的工程基准，验证了无标记视觉反馈支撑零样本迁移的可行性。对于具身智能开发者，这降低了DLO操作对高精度模型与真机数据的依赖，表明通过优化仿真器并行度与观测空间设计，可有效跨越Sim-to-Real鸿沟，加速非刚性物体操作算法的落地验证。

## 英文摘要 (Abstract)

Deformable linear object (DLO) manipulation is challenging because high-dimensional configurations, nonlinear contact dynamics, and diverse material and geometric properties produce a broad range of object responses that are both difficult to model and data-intensive to learn. This work studies reinforcement learning for planar manipulation of a one-end-grasped DLO, focusing on training efficiency, observation design, robustness, and zero-shot sim-to-real transfer. We compare three Proximal Policy Optimization training stacks and find that the GPU-parallel MuJoCo Playground stack enables faster convergence and more consistent performance than CPU-based alternatives under the same wall-clock budget. Relative keypoint observations and an asymmetric actor-critic formulation further improve learning efficiency. Policies trained with observation noise, delay, domain randomization, and disturbances achieve simulation success rates of 96.4% for single-point tracking, 98.0% for disturbance recovery, and 88.8% for two-point tracking. Using markerless RGB-D keypoint estimation, the policies transfer without real-world fine-tuning, succeeding in 9/10, 10/10, and 8/10 physical trials, respectively.

作者：Vinay Rodrigues, Sara Zeynalpour, Gokhan Solak, Sebastian Hjorth, Arash Ajoudani
:::

## 原始参考来源 (Sources)

- [arXiv 2611.07261：An Empirical Study of Reinforcement Learning for Keypoint Positioning of Deformable Linear Objects](https://arxiv.org/abs/2611.07261)