外观
论文arXiv 2610.2154813 天前
部分可观测下受约束强化学习的有限时间保证
原文链接：Finite-Time Guarantees for Constrained Reinforcement Learning under Partial Observability（arXiv 2610.21548 · 发布日期：2026-10-15）
Finite-Time Guarantees for Constrained Reinforcement Learning under Partial Observability
针对CPOMDP中奖励最大化与成本约束问题，采用有限历史窗口和原始-对偶算法，并在部分可观测网格世界验证。
核心要点
- 部分可观测下，最优策略可能依赖任意长的观测-动作历史，使规划和学习更具挑战。
- 将有限历史窗口表示为受约束MDP的状态，并开发原始-对偶算法处理学习问题。
- 在滤波稳定性假设下，证明有限状态模型与原CPOMDP的最优性和可行性误差随窗口长度几何衰减。
- 部分可观测网格世界实验展示了该方法的有效性。
验证范围与局限：实验验证范围为部分可观测网格世界。
英文摘要 (Abstract)
We study reinforcement learning in constrained partially observable Markov decision processes (CPOMDPs), where an agent maximizes reward subject to a cost constraint using only incomplete information about the underlying state. The lack of state observability may require optimal policies to depend on arbitrarily long observation-action histories, making planning and learning challenging. We leverage recent advances in finite-memory approximations of POMDPs and represent finite history windows as states of a constrained MDP. We then develop a primal--dual algorithm that provides the first non-asymptotic guarantees that jointly bound average reward suboptimality and constraint violation in the original CPOMDP. A key challenge is that this finite-window approximation affects both reward values and the set of feasible policies. Under a filter-stability assumption, we derive bounds relating optimality and feasibility in the finite-state model to those in the original CPOMDP, with approximation errors that decay geometrically with the window length. Experiments in partially observable gridworlds demonstrate the effectiveness of our approach.
作者：Valentina Zaccaria, Philip Jordan, Tingting Ni, Gian Antonio Susto, Maryam Kamgarpour
来源
本页事实对应的公开页面，以原文为准。