论文/arXiv 2610.09218·22 天前

RLDISCOVER：LLM驱动的强化学习算法协同进化

RLDiscover用LLM驱动的渐进协同进化与概率评估，自演化无模型深度RL算法，并在SAC、PPO、DQN四个基准套件验证提升。

AI 整理

• 本体 SDK 与控制
• 具身模型
• 评测与安全

原文链接：RLDISCOVER: LLM-driven co-evolution of reinforcement learning algorithms（arXiv 2610.09218 · 发布日期：2026-10-06）

RLDISCOVER: LLM-driven co-evolution of reinforcement learning algorithms

RLDiscover用LLM驱动的渐进协同进化与概率评估，自演化无模型深度RL算法，并在SAC、PPO、DQN四个基准套件验证提升。

核心要点

• 动机：联合搜索耦合算法组件难扩展，候选算法训练昂贵且跨随机种子适应度不确定。
• 方法：Progressive Co-Evolution从定向组件编辑推进到联合进化，处理组件依赖问题。
• 评估：Progressive Probabilistic Evaluation以分阶段训练和重复评估平衡搜索广度与保真度。
• 结果：四个基准套件上平均回报提升，族内中位增益32%-84%，所选程序可迁移到未见任务。

验证范围与局限：验证范围为SAC、PPO、DQN在四个基准套件上的实验，以及测量的SAC运动运行和未见任务迁移。

英文摘要 (Abstract)

LLM-guided program evolution has enabled discoveries in mathematics and computational optimization, raising the prospect of reinforcement learning (RL) algorithms that self-evolve to improve how agents learn. However, realizing this prospect faces two obstacles. Joint search over coupled algorithmic components is difficult to scale: simultaneous changes can disrupt learning, while isolated changes overlook their dependencies. Evaluating candidate algorithms also requires costly training, with fitness remaining uncertain across random seeds. We introduce RLDiscover, a framework for the self-evolution of model-free deep RL algorithms. Progressive Co-Evolution advances from targeted component edits to joint evolution, while Progressive Probabilistic Evaluation balances search breadth and evaluation fidelity through staged training and repeated evaluation. Experiments across SAC, PPO, and DQN on four benchmark suites show substantial improvements in mean return, with per-family median gains of 32%-84% and a peak return ratio of approximately 363x over a near-zero baseline. These gains include transitions from failed learning to successful task completion, and improvements persist when evolution starts from stronger open-source implementations. On measured SAC locomotion runs, evaluation uses approximately one-fifteenth the estimated compute required to fully evaluate the same candidate pool. Remarkably, independent searches repeatedly discover interpretable combinations of adaptive robust losses, progress-dependent value targets, and running statistics, with selected programs transferring to unseen tasks. These findings point toward a broader role for self-evolution in AI: discovering interpretable algorithms that improve how agents learn.

作者：Haoran Li, Zengle Ge, Xiaomin Yuan, Yui Lo, Songlin Zhou, Jiahua Ying, Haoxin Li, Qianhui Liu, Yuanhang Liu, Jiaqun Liu

来源

本页事实对应的公开页面，以原文为准。

1. arXiv 2610.09218：RLDISCOVER: LLM-driven co-evolution of reinforcement learning algorithms访问 2026-10-21