# 面向富接触操作强化学习的势场动作表示

> 原文链接：https://robotics.hellobike.com/papers/2026-09-18-2609-21609
> 频道：papers | 更新日期：2026-09-18 | 标签：arxiv, cs.RO
> 核心摘要：PA-RL用人工势场动作表示减轻富接触操作强化学习的低层运动负担，并在仿真与真机插孔中验证。

# 面向富接触操作强化学习的势场动作表示

::: v-pre
> 原文链接：[Potential-Field Action Representation for Reinforcement Learning in Contact-Rich Manipulation](https://arxiv.org/abs/2609.21609)（arXiv 2609.21609 · 发布日期：2026-09-18）

**Potential-Field Action Representation for Reinforcement Learning in Contact-Rich Manipulation**

PA-RL用人工势场动作表示减轻富接触操作强化学习的低层运动负担，并在仿真与真机插孔中验证。

## 核心要点

- 无模型强化学习可学富接触技能，但直接笛卡尔命令会增加策略的低层运动学习负担。
- PA-RL让策略调节类能量势场参数，由势场产生状态相关引导方向并经笛卡尔阻抗控制执行。
- 插孔仿真中用同一RL算法对比三类动作空间，PA-RL在限时训练内达100%成功率。
- 相较最佳基线，PA-RL降低关节力矩和笛卡尔加速度变化；仿真策略无需微调完成9/9次真机插入。

**验证范围与局限**：验证范围为插孔任务：仿真对比多种动作空间，并进行9次真机插入部署测试。

## 英文摘要 (Abstract)

Model-free reinforcement learning can acquire contact-rich robotic manipulation skills through trial-and-error interaction, but it often requires the policy to learn both task strategy and low-level motion generation. In this setting, the action representation is critical because it determines how policy outputs are converted into robot motion, shaping both exploration and physical execution. Direct Cartesian command interfaces require the policy to generate motion at every decision step, coupling task-level adaptation with continuous low-level control and increasing the learning burden. We propose PA-RL, a reinforcement-learning framework that uses artificial potential fields as the action representation. Instead of commanding motion directly, the policy adapts the parameters of an energy-like potential field, which generates a state-dependent guidance direction executed through a Cartesian impedance controller. We evaluate PA-RL on peg-in-hole insertion, a representative contact-rich task with nonlinear dynamics and discontinuous contact transitions. In simulation, PA-RL is compared with Cartesian velocity, Cartesian pose, and variable-impedance action spaces using the same RL algorithm. PA-RL is the only method to reach a 100% evaluation success rate within the allotted training time, while the best baseline reaches 92.6%. It also reduces joint-torque variation by 55.4% and Cartesian acceleration variation by 70.8% relative to the best baseline, without explicit motion-quality penalties in the reward. The simulation-trained policy further completes 9/9 real-robot insertions without fine-tuning, demonstrating the deployment feasibility of the learned potential-field interface.

作者：Xinyu Liu, Gökhan Solak, Arash Ajoudani
:::

## 原始参考来源 (Sources)

- [arXiv 2609.21609：Potential-Field Action Representation for Reinforcement Learning in Contact-Rich Manipulation](https://arxiv.org/abs/2609.21609)