# 高阶动作监督造就强大的策略类

> 原文链接：https://robotics.hellobike.com/papers/2026-10-08-2610-11175
> 频道：papers | 更新日期：2026-10-08 | 标签：arxiv, cs.RO
> 核心摘要：针对IL/RL仅监督动作标签导致的控制不稳定，提出零阶与一阶动作联合监督，并在OGBench、D4RL验证鲁棒性提升。

# 高阶动作监督造就强大的策略类

::: v-pre
> 原文链接：[Higher-Order Action Supervision Makes A Strong Policy Class](https://arxiv.org/abs/2610.11175)（arXiv 2610.11175 · 发布日期：2026-10-08）

**Higher-Order Action Supervision Makes A Strong Policy Class**

针对IL/RL仅监督动作标签导致的控制不稳定，提出零阶与一阶动作联合监督，并在OGBench、D4RL验证鲁棒性提升。

## 核心要点

- 现实机器人和自动驾驶中，IL/RL常因只优化零阶动作而出现控制不稳定与鲁棒性问题。
- 方法以带理论保证的损失方案，同时监督动作及其一阶动态，无需修改现有策略结构。
- 该模块可接入确定性、随机或flow策略，并作为轻量插件融入离线RL框架。
- 在OGBench和D4RL连续控制环境中，性能、鲁棒性及低数据OOD泛化均有提升。

**验证范围与局限**：验证范围为OGBench和D4RL连续控制环境，摘要未报告真实系统实验。

## 英文摘要 (Abstract)

Modern data-driven decision-making methods, such as imitation learning (IL) and reinforcement learning (RL), have achieved great success in solving many complex tasks. However, these methods often suffer from serious control instability and robustness issues when applied in real-world applications such as robotics and autonomous driving, posing notable challenges for their practical deployment. We argue that this instability issue stems largely from their limitations in solely supervising and optimizing zeroth-order actions (i.e., the action labels), failing to account for higher-order action dynamics and temporal consistency. In this paper, we show that simultaneously supervising both zeroth- and first-order actions can dramatically enhance policies' performance and control robustness. To achieve this, we introduce a novel and elegant loss scheme supported by formal theoretical guarantees that can equip any off-the-shelf policy model (e.g., deterministic, stochastic, or flow policies) with the capability for higher-order action supervision, without requiring any structural modifications. Moreover, our proposed method can serve as a lightweight plug-and-play module that seamlessly integrates with a broad spectrum of existing offline RL frameworks. Extensive evaluations on OGBench and D4RL demonstrate that our approach yields substantial performance and robustness improvements across a wide range of continuous control environments. Notably, our method can also enhance policies' out-of-distribution (OOD) generalization capability in the challenging low-data regime, making it an ideal tool in tackling many real-world control problems.

作者：Peng Cheng, Yunxian Hou, Zhi Zhou, Qian Zhang, Chang Huang, Xianyuan Zhan
:::

## 原始参考来源 (Sources)

- [arXiv 2610.11175：Higher-Order Action Supervision Makes A Strong Policy Class](https://arxiv.org/abs/2610.11175)