外观
论文arXiv 2609.217351 个月前
GEM-MPC：通过专家引导规划平衡探索与利用
原文链接：GEM-MPC: Balancing Exploration and Exploitation through Expert-Guided Planning（arXiv 2609.21735 · 发布日期：2026-09-18）
GEM-MPC: Balancing Exploration and Exploitation through Expert-Guided Planning
GEM-MPC针对高维连续控制探索难题，结合MPPI规划、克隆规划器策略与KL正则探索策略，并在连续控制基准上验证优于规划式基线。
核心要点
- 高维连续控制中的有效探索困难，规划式方法训练中易出现组件错位。
- GEM-MPC用MPPI结合克隆规划器的策略与KL正则探索策略。
- 门控先验蒸馏仅在存储规划分布优于当前先验时学习，缓解陈旧数据影响。
- 在连续控制基准上，在较低计算预算下持续优于现有规划式基线。
验证范围与局限：验证范围为连续控制基准，摘要未说明真机实验或具体任务细节。
英文摘要 (Abstract)
Effective exploration in high-dimensional continuous control remains a central challenge in reinforcement learning. Planning-based methods address this by combining online planning with learned policies and value functions, but their components can become misaligned during training: learned sampling policies may diverge from planner behavior, while planning distributions stored in replay become stale as the model and value function evolve. Reanalysis can refresh these targets, but at substantial computational cost. We propose GEM-MPC, an MPPI-based reinforcement learning method that improves the interaction between planning and learning. GEM-MPC uses MPPI to combine a policy trained to clone the planner with a KL-regularized policy that explores around it, providing complementary exploitation and guided exploration within planning. We further introduce Gated Prior Distillation, which selectively learns from stored planning distributions only when they provide a better target than the current prior, reducing the impact of stale planning data without requiring full reanalysis. Across continuous-control benchmarks, GEM-MPC consistently outperforms existing planning-based baselines under lower computational budgets.
作者：Álvaro Serra-Gómez, Thomas M. Moerland
来源
本页事实对应的公开页面，以原文为准。