外观
论文arXiv 2610.1988314 天前
通过 Drifting 进行占用建模：一步生成实现直接策略提取
原文链接：Occupancy Modeling via Drifting: One-Step Generation Enables Direct Policy Extraction（arXiv 2610.19883 · 发布日期：2026-10-14）
Occupancy Modeling via Drifting: One-Step Generation Enables Direct Policy Extraction
该文面向从无标签数据强化学习，采用 Drifting 一步生成采样器建模占用，并在标准基准上验证可直接提取策略。
核心要点
- 目标是在无奖励轨迹上预训练占用生成模型，待少量奖励标注数据可用时提取策略。
- 论文从占用多模态刻画、生成模型类别和策略提取方法三方面分解并比较选择。
- 提出用 Drifting 建模折扣状态访问占用，并通过占用模型直接微分提取策略。
- 实验验证各项设计，并观察到该简化设置在标准基准上带来性能提升。
验证范围与局限：验证范围为标准基准，摘要未说明具体任务或是否包含真机实验。
英文摘要 (Abstract)
The growing interest in behavior foundation models, combined with recent advancements in generative modeling, has led to an appealing recipe for reinforcement learning from unlabeled data: pre-train a generative model to model occupancies, i.e., discounted state visitations, on reward-free trajectories, such that a policy can be easily extracted once a small reward-labeled dataset becomes available. We decompose this recipe along three axes: (i) the degree to which the multi-modal structure of occupancies must be captured, (ii) the class of generative model, and (iii) the method used for policy extraction. Along each axis, we consider existing choices, propose simpler alternatives, and validate each decision experimentally. This analysis culminates in a novel, streamlined method that leverages drifting, a modern one-step generative sampler, to model the occupancy. We show that this enables policy extraction via direct differentiation through the occupancy model, and bypasses the need for critic distillation. Overall, we observe that this nimble setup leads to improvements in performance on standard benchmarks.
作者：Robin Uhrich, Marco Bagatella, Núria Armengol Urpí, Pavel Kolev, Georg Martius
来源
本页事实对应的公开页面，以原文为准。