外观
论文arXiv 2609.362381 个月前
ChronoSRL：用于自监督强化学习的时间几何
原文链接：ChronoSRL: Temporal Geometry for Self-Supervised Reinforcement Learning（arXiv 2609.36238 · 发布日期：2026-09-28）
ChronoSRL: Temporal Geometry for Self-Supervised Reinforcement Learning
ChronoSRL为critic嵌入引入时间几何，并在七个运动与导航基准及四足机器人仿真到现实任务中验证。
核心要点
- 空间上接近的目标可能因障碍、地形或智能体能力限制而在时间上遥远。
- 方法训练状态-动作与目标嵌入距离匹配到达时间，并推远未到达或异轨迹目标。
- 基于生存强化学习，从时间嵌入预测到达时间分布和在目标附近停留时间。
- 在七个标准运动与导航基准上学习更快、性能更高；在四足机器人任务中表现优于被测自监督方法。
验证范围与局限：验证范围包括七个标准运动与导航基准，以及四足机器人真实感仿真到现实速度跟踪、目标位置到达和爬箱任务。
英文摘要 (Abstract)
A goal that is close in space can be far away in time. Obstacles, terrain, and the agent's own capabilities determine how long it takes to get there. Yet, critics in contrastive and survival reinforcement learning do not measure the distances in their representation space in units of time. We therefore introduce ChronoSRL, which gives the critic's embeddings an explicit temporal geometry. The distance between state-action and goal embeddings is trained to match the time that the agent takes to reach the goal (goal-reaching time), while goals that were not reached, and goals from other trajectories, are pushed at least one discount horizon away. Furthermore, reaching a goal quickly once does not mean that reaching it is reliable in general, so the policy should not follow the temporal distance directly. Instead, we build on survival reinforcement learning and predict from our temporal embeddings not only the full distribution of goal-reaching times but also the time spent near the goal. Thereby, the policy is trained to favor actions that reach the goal sooner and more reliably and that keep the agent near it. ChronoSRL learns faster and reaches higher performance than contrastive, action-chunked contrastive, and survival reinforcement learning baselines on seven standard locomotion and navigation benchmarks, even with much smaller networks. To test the limits of self-supervised reinforcement learning, we introduce velocity tracking, goal-position reaching, and box climbing tasks with a quadruped robot in a realistic sim-to-real locomotion setup, and show how the shaping terms that are typical for robotics can be naturally incorporated into our framework. ChronoSRL is the only one of the tested self-supervised reinforcement learning methods that learns to stay at the commanded velocities and goal positions, and climbs the highest boxes.
作者：Nico Bohlinger, Jan Peters
来源
本页事实对应的公开页面，以原文为准。