外观
论文arXiv 2610.0898922 天前
LASER：用于支持约束熵正则化离线RL的潜在空间伴随匹配
原文链接：LASER: Latent Space Adjoint Matching for Support-Constrained Entropy-Regularized Offline RL（arXiv 2610.08989 · 发布日期：2026-10-06）
LASER: Latent Space Adjoint Matching for Support-Constrained Entropy-Regularized Offline RL
LASER针对离线RL的OOD动作风险，在受限潜在空间引入伴随匹配与熵正则，并在40个OGBench任务上验证。
核心要点
- 离线RL可免在线交互优化策略，但执行OOD动作仍是主要瓶颈。
- LASER在行为克隆流策略的受限潜在空间中进行熵正则化RL。
- 潜在空间伴随匹配支持表达性流策略，并避免通过时间反向传播。
- 在40个不同数据质量的OGBench任务上达到SOTA，并给出项目网站。
验证范围与局限：验证范围为40个具有不同数据质量的OGBench任务。
英文摘要 (Abstract)
While offline reinforcement learning (RL) enables policy optimization from static datasets without costly online interaction, it remains bottlenecked by the risk of executing out-of-distribution (OOD) actions. Recent approaches mitigate this by learning a behavior-cloning policy through flow matching and then performing RL within its constrained latent space. However, naively optimizing the latent policy can easily cause the policy to collapse into a brittle mode or exploit sharp artifacts of the learned critic. In this work, we find that entropy regularization is essential in latent-space RL for addressing these challenges. We introduce LASER, a novel offline RL algorithm that applies latent-space adjoint matching to achieve entropy-regularized latent-space RL with expressive flow policies while avoiding backpropagation through time. Through comprehensive experiments on 40 challenging OGBench tasks with varying dataset qualities, we show that LASER achieves state-of-the-art performance. Notably, LASER uses fixed method-specific hyperparameters across all tasks and outperforms the evaluated baselines, including those with task- and dataset-specific tuning, which highlights the robust applicability of LASER. Project website: https://mit-realm.github.io/laser/.
作者：Songyuan Zhang, Oswin So, Eric Yang Yu, Matthew Cleaveland, Peter Crowley-Dolen, Chuchu Fan
来源
本页事实对应的公开页面，以原文为准。