# FoSeRL：强化学习策略的形式化序列鲁棒性认证

> 原文链接：https://robotics.hellobike.com/papers/2026-10-03-2610-04754
> 频道：papers | 更新日期：2026-10-03 | 标签：arxiv, cs.RO
> 核心摘要：FoSeRL无需改动策略，利用可重置仿真器认证随机环境中稀疏动作攻击下的回报损失，并在连续控制环境验证。

# FoSeRL：强化学习策略的形式化序列鲁棒性认证

::: v-pre
> 原文链接：[FoSeRL: Formal Sequential Robustness Certification for Reinforcement Learning Policies](https://arxiv.org/abs/2610.04754)（arXiv 2610.04754 · 发布日期：2026-10-03）

**FoSeRL: Formal Sequential Robustness Certification for Reinforcement Learning Policies**

FoSeRL无需改动策略，利用可重置仿真器认证随机环境中稀疏动作攻击下的回报损失，并在连续控制环境验证。

## 核心要点

- 少量动作扰动也会显著削弱部署策略，随机环境中区分攻击损失与自然回报波动很难。
- 框架保持部署策略不变，不做平滑或重训，依赖支持共享随机性和一步查询的可重置仿真器。
- 通过共享初始状态和随机性，并把运行回报差作为状态坐标，将轨迹级认证化为终端安全问题。
- 在六个随机连续控制环境和TD3、SAC、PPO上，认证非平凡鲁棒性前沿且预算显著大于策略平滑。

**验证范围与局限**：验证范围为六个随机连续控制环境中的TD3、SAC和PPO策略。

## 英文摘要 (Abstract)

Even a few action perturbations can substantially degrade the performance of a deployed decision policy. Certifying the resulting return loss is challenging in stochastic environments, where returns vary even without an attack. We introduce FoSeRL, a framework for certifying deployed RL policies against precommitted, temporally sparse action attacks. The deployed policy is unchanged, with no smoothing or retraining. Certification requires a resettable simulator supporting shared randomness and independent one-step successor queries, but no analytical dynamics model. FoSeRL certifies that an attacked episode loses no more than a prescribed amount of return relative to the same episode unattacked, with at least a target probability and at a user-specified confidence level. Both runs share the initial state and randomness, so the measured loss reflects the attack, not the episode; carrying the running return gap as a state coordinate makes it the terminal value, reducing trajectory-level certification to terminal safety. Time-dependent barrier conditions on the augmented state bound the terminal failure probability: satisfied exactly, they certify every admissible precommitted attack; learned from sampled trajectories and verified on held-out data, they certify the same guarantee under a specified attack-episode setting. Across six stochastic continuous-control environments and three RL policy families (TD3, SAC, and PPO), FoSeRL certifies non-trivial cardinality--magnitude robustness frontiers, achieves substantially larger certified budgets than policy smoothing, and reveals marked robustness differences among policies with comparable nominal performance.

作者：Sara Taheri, Deep Kumar Ganguly, Jan Křetínský, Majid Zamani
:::

## 原始参考来源 (Sources)

- [arXiv 2610.04754：FoSeRL: Formal Sequential Robustness Certification for Reinforcement Learning Policies](https://arxiv.org/abs/2610.04754)