具身开发者社区

外观

论文arXiv 2610.2191213 天前

战略博弈中的奖励学习 ​

本文研究战略多智能体中奖励可识别性，提出主动IRL算法，并在反偷猎场景验证奖励恢复与策略性能。

AI 整理
  • 具身模型

原文链接：Learning Rewards in Strategic Games（arXiv 2610.21912 · 发布日期：2026-10-15）

Learning Rewards in Strategic Games

本文研究战略多智能体中奖励可识别性，提出主动IRL算法，并在反偷猎场景验证奖励恢复与策略性能。

核心要点 ​

  • 多智能体战略环境中，仅解释观测行为的奖励未必能诱导最优决策。
  • 以战略可识别性分析，证明固定策略对即使样本增多也可能不足。
  • 提出面向两玩家重复Stackelberg博弈的主动IRL，自适应选择领导者策略以获取信息。
  • 结合最大似然与实验设计，给出有限样本次优性保证，并在反偷猎场景展示收益。

验证范围与局限：验证包括理论分析及一个逼真的反偷猎设置评估，范围为两玩家重复Stackelberg博弈。

英文摘要 (Abstract) ​

Understanding the objectives that drive agents' behavior is crucial in strategic environments, where multiple agents interact and adapt to one another. This challenge arises in domains such as cybersecurity, autonomous driving, and online markets, where agents' underlying incentives are typically unknown and must be inferred from their observed behavior. Inverse reinforcement learning (IRL) provides a natural framework for this problem, but extending IRL to strategic multi-agent settings introduces a fundamental challenge: a reward that explains observed behavior does not necessarily induce optimal strategies when used for decision-making. In this work, we study this problem through the lens of \emph{strategic identifiability}, and show that learning from a fixed policy pair can be insufficient to strategically identify rewards, independent of the number of expert samples $K$. Motivated by this, we introduce an active IRL algorithm for two-player repeated Stackelberg games, in which the leader adaptively selects policies to elicit informative responses about the follower's reward. Combining maximum-likelihood estimation with experimental design, we derive finite-sample guarantees on the Stackelberg sub-optimality induced by the learned reward, matching up to polylogarithmic factors a new minimax learning rate of $K^{-1/2}$. Finally, we evaluate our approach in a realistic anti-poaching setting, demonstrating the benefits of active exploration for reward recovery and downstream strategic performance.

作者：Antoine Bergerault, Cyrille Kone, Giorgia Ramponi

来源

本页事实对应的公开页面，以原文为准。

  1. arXiv 2610.21912：Learning Rewards in Strategic Games访问 2026-10-21