外观
论文arXiv 2610.0436425 天前
迭代式奖励设计的鸟瞰视角
原文链接：A Bird's-Eye View of Iterative Reward Design（arXiv 2610.04364 · 发布日期：2026-10-03）
A Bird's-Eye View of Iterative Reward Design
提出BIRD统一基准，用匹配反馈与训练预算比较迭代奖励设计方法，并在MuJoCo等环境验证简单设计选择可提升性能。
核心要点
- 强化学习奖励函数设计通常依赖专业经验和反复试错，现有LLM迭代方法因实现、反馈和环境差异难以比较。
- BIRD将现有方法表达到统一的配置与评估空间，支持直接比较、消融设计选择并原型化新组件。
- 在MuJoCo、Meta-World、Assistax和HumanoidBench上，少量简单设计选择被发现能持续改善性能。
- 组合这些设计选择后，性能显著优于所评估的既有方法；代码已在GitHub开源。
验证范围与局限：验证范围为MuJoCo、Meta-World、Assistax和HumanoidBench中的评估环境。
英文摘要 (Abstract)
Designing effective reward functions in RL typically requires substantial expertise and trial and error. Recent work automates this process with LLM-based systems that generate and iteratively improve reward code using policy feedback. However, these methods are often hard to compare because they differ in implementation details, feedback assumptions, and evaluation environments. To address this, we introduce a Benchmark for Iterative Reward Design (BIRD) that expresses existing methods in a unified configuration and evaluation space. This lets us compare algorithms directly, ablate individual design choices, and prototype new components under matched feedback conditions and policy-training budgets. Across MuJoCo, Meta-World, Assistax, and HumanoidBench, we identify a small set of simple design choices that consistently improve performance. Combining these choices yields significantly better performance than the evaluated methods from prior work. Our results highlight the strength of simple baselines and motivate further study of when additional algorithmic complexity improves iterative reward design. Code is available at https://github.com/safety-research/bird.
作者：Logan Mondal Bhamidipaty, Lauren Robson, Linda Petrini, Shengrui Lyu, Kamal Ndousse
来源
本页事实对应的公开页面，以原文为准。