外观
论文arXiv 2610.0084927 天前
为目标条件强化学习学习多重时间尺度
原文链接：Learning Multiple Timescales for Goal-Conditioned Reinforcement Learning（arXiv 2610.00849 · 发布日期：2026-10-01）
Learning Multiple Timescales for Goal-Conditioned Reinforcement Learning
针对离线GCRL长时程任务信号不足，GITA以k条件化价值函数聚合多尺度监督，并在OGBench验证提升。
核心要点
- 离线目标条件强化学习在长时程任务中受折扣影响，远距离状态价值差异易被近似误差淹没。
- GITA将时间抽象尺度k作为价值函数条件，用单一价值函数同时保留局部与长程信号。
- 其通过跨多个k聚合优势加权监督训练一个策略，正优势更大的尺度对更新贡献更强。
- 在OGBench上，GITA超过多种离线GCRL基线，并优于最强固定k方法OTA。
验证范围与局限：验证范围为OGBench上的离线GCRL任务。
英文摘要 (Abstract)
Existing approaches to offline goal-conditioned reinforcement learning (GCRL) struggle with long-horizon tasks. Discounting shrinks value differences between distant states until they fall below the function approximation error, leaving the agent with no signal for ranking states. Temporal abstraction, which treats k environment steps as a single transition, restores this signal at long range, but no single fixed k suits all state-goal distances: large k preserves value differences across long temporal distances while collapsing distinctions between nearby states, and small k does the reverse. We make this trade-off explicit and introduce Generalized Implicit Temporal Abstraction (GITA), which conditions a single value function on k. GITA trains one policy by aggregating advantage-weighted supervision across multiple k values, so scales assigning larger positive advantages to a state-goal pair contribute more strongly to its update. GITA does not need to choose between local resolution and long-range signal; it retains both without committing to a single k. On OGBench, GITA outperforms a broad range of offline GCRL baselines, raising average success rate across all tasks by 25 percentage points (73% relative improvement) over HIQL. It also improves over the strongest fixed-k method, OTA, by 7 percentage points (14% relative).
作者：Pedro Robles Dutenhefner, Dikshant Shehmar, Wagner Meira, Marlos C. Machado
来源
本页事实对应的公开页面，以原文为准。