# 带反馈校正的动作分块近端策略优化

> 原文链接：https://robotics.hellobike.com/papers/2026-09-28-2609-36250
> 频道：papers | 更新日期：2026-09-28 | 标签：arxiv, cs.RO
> 核心摘要：面向高维机器人控制中的动作分块学习与反馈不足，提出ACPPO及在线校正器，并在25个仿真任务验证。

# 带反馈校正的动作分块近端策略优化

::: v-pre
> 原文链接：[Action Chunking Proximal Policy Optimization with Feedback Correction](https://arxiv.org/abs/2609.36250)（arXiv 2609.36250 · 发布日期：2026-09-28）

**Action Chunking Proximal Policy Optimization with Feedback Correction**

面向高维机器人控制中的动作分块学习与反馈不足，提出ACPPO及在线校正器，并在25个仿真任务验证。

## 核心要点

- 现有动作分块方法常依赖块级价值函数，且块内开环执行削弱接触任务反应性。
- ACPPO采用分块actor与标准状态价值critic，避免学习分块Q函数。
- ACPPO-Corr加入逐步反馈校正器，在每个动作块内在线调整规划动作。
- 在25个仿真机器人任务中ACPPO-Corr总体最优；消融显示中等块长和正则化重要，代码已开源。

**验证范围与局限**：验证范围为IsaacGym和Bi-DexHands中的25个仿真任务，涵盖运动、机械臂操作和灵巧手-物体交互。

## 英文摘要 (Abstract)

Action chunking provides temporal abstraction in reinforcement learning by selecting short action sequences instead of individual actions, but many existing approaches face two limitations in high-dimensional robotic control. First, many rely on value functions over action chunks, which can be difficult to learn as action dimensionality and chunk length grow. Second, executing chunks open-loop removes within-chunk feedback, limiting reactivity in contact-rich tasks. We present Action Chunking PPO (ACPPO), a PPO extension that uses a chunked actor while retaining a standard state-value critic, thereby avoiding chunked Q-functions. We further propose ACPPO-Corr, which augments the chunk planner with a stepwise feedback corrector that adjusts planned actions online within each chunk. Across 25 simulated robotics tasks from IsaacGym and Bi-DexHands, spanning locomotion, arm manipulation, and dexterous hand-object interaction, ACPPO-Corr achieves the strongest aggregate performance among evaluated methods and performs best on both decision-frequency-sensitive and decision-frequency-neutral task subsets. Ablations show that moderate chunk lengths work best and that corrector regularization is important for balancing chunk-level planning with local feedback. These results suggest that action chunking can be effective in online PPO when chunk-level planning is paired with closed-loop correction. The code is available at: https://github.com/hshhahn/ACPPO.

作者：Sanghyun Hahn, Jonghyun Choi
:::

## 原始参考来源 (Sources)

- [arXiv 2609.36250：Action Chunking Proximal Policy Optimization with Feedback Correction](https://arxiv.org/abs/2609.36250)