外观
论文arXiv 2609.356151 个月前
用于质量多样性的行为基础模型
原文链接：Behavioral Foundation Models for Quality Diversity（arXiv 2609.35615 · 发布日期：2026-09-28）
Behavioral Foundation Models for Quality Diversity
该工作用BFM紧凑潜在空间进行QD搜索，在连续控制基准上发现多样且高性能策略。
核心要点
- QD通常在高维策略参数空间搜索，目标是发现多样且高性能的策略库。
- BFM-QD改在BFM的紧凑潜在行为空间中执行质量多样性搜索。
- 框架给出闭式无梯度策略改进算子，近似策略梯度且无需critic训练或反传。
- 在密集运动、稀疏导航和接触丰富操作的连续控制基准上，持续优于参数空间基线。
验证范围与局限：验证范围为连续控制基准中的密集运动、稀疏导航和接触丰富操作任务。
英文摘要 (Abstract)
Behavioral Foundation Models (BFMs) are an emerging paradigm in reinforcement learning, playing a role analogous to large language models in natural language processing: they have shown remarkable versatility, enabling zero-shot performance, fast imitation, and online adaptation, all by exploiting the structure of a latent space. In this work, we investigate whether the latent behavioral space induced by BFMs can serve as an effective search space to discover large repertoires of behaviorally diverse and high-performing policies through Quality-Diversity (QD) methods. While QD methods generally search directly in high-dimensional policy parameter space, in this paper, we present BFM-QD, a framework that performs QD search in the compact latent space of a BFM. We further show that the BFM-QD framework provides a closed-form, gradient-free policy improvement operator that approximates a policy gradient update, but requires no critic training and no backpropagation. Across continuous-control benchmarks spanning dense locomotion, sparse navigation, and contact-rich manipulation, BFM-QD consistently outperforms parameter-space baselines, with particularly stark gains in sparse and deceptive settings, where all tested parameter-space QD methods collapse to near-zero performance. These results show the effectiveness of the BFM-QD framework, benefiting from the synergy between dimensionality reduction of the search space and offline pretraining from diverse behavioral data. This positions BFMs as a general-purpose backbone for QD optimization, extending their utility beyond zero-shot task solving to the discovery of diverse behavioral repertoires.
作者：Nazim Bendib, Nicolas Perrin, Olivier Sigaud
来源
本页事实对应的公开页面，以原文为准。