外观
论文arXiv 2610.1440019 天前
Sparse-TD-JEPA：用稀疏潜在任务表示改进零样本强化学习
原文链接：Sparse-TD-JEPA: Improving Zero-Shot Reinforcement Learning With Sparse Latent Task Representations（arXiv 2610.14400 · 发布日期：2026-10-09）
Sparse-TD-JEPA: Improving Zero-Shot Reinforcement Learning With Sparse Latent Task Representations
针对密集任务表示导致奖励弥散的问题，Sparse-TD-JEPA用l1稀疏惩罚改进TD-JEPA，并在DMC与OGBench验证。
核心要点
- 密集任务编码会使推断奖励在状态空间中弥散，削弱目标相关状态对比。
- 方法在训练时加入l1稀疏惩罚，将奖励预测结构集中到更小潜在子空间。
- 该改动无需架构变化，也不修改测试时的闭式线性回归推断流程。
- 在13个域级比较均优于TD-JEPA，9个域达到既有零样本基线最高均值表现。
验证范围与局限：验证范围为DMC与OGBench中的65个任务，涵盖运动、导航、轨迹拼接、解谜和机器人操作。
英文摘要 (Abstract)
Successor-feature-based zero-shot reinforcement learning methods pre-train a task encoder and successor-feature predictor on offline, reward-free data, then recover a policy at test time by solving a closed-form linear regression over a small labeled dataset. We identify an underexplored limitation of this approach: when the task encoder produces dense representations, the inferred reward signal becomes diffuse across the state space, reducing the contrast between goal-relevant states and the background and providing a weaker signal for the policy to follow. We address this limitation by introducing an l1 sparsity penalty that concentrates reward-predictive structure into a smaller effective subspace of the task encoder's latent space. The penalty is applied only during training and requires no architectural change or modification to the closed-form inference procedure. We instantiate this approach as Sparse-TD-JEPA, a single-line of code addition based on Temporal-Difference Joint-Embedding Predictive Architecture (TD-JEPA) that significantly improved its performance in zero-shot reinforcement learning. We evaluate Sparse-TD-JEPA on 65 tasks across the DeepMind Control (DMC) Suite and OGBench, covering locomotion, navigation, trajectory stitching, puzzle solving, and robotic manipulation. Sparse-TD-JEPA improves over TD-JEPA in all 13 domain-level comparisons. On DMC, it increases the average return from 661 +- 6 to 714 +- 3. On OGBench, it increases the average success rate from 38.0 +- 0.8 to 48.4 +- 2.3, with improvements of 13.4-18.0 percentage points on the evaluated antmaze navigation and stitching domains. Overall, Sparse-TD-JEPA achieves the highest mean performance among established zero-shot baselines in nine of the 13 domains. Analysis further shows that sparsity produces a sharper and more task-informative inferred reward landscape.
作者：Duaa Nakshbandi, Zhiyu An, Wan Du
来源
本页事实对应的公开页面，以原文为准。