外观
论文arXiv 2609.306961 个月前
基于视觉的敏捷缝隙穿越学习：使用预热 Critic 的可微仿真
原文链接：Learning Vision-Based Agile Gap Traversal: Differentiable Simulation with a Warm-Started Critic（arXiv 2609.30696 · 发布日期：2026-09-25）
Learning Vision-Based Agile Gap Traversal: Differentiable Simulation with a Warm-Started Critic
提出两阶段强化学习框架，用可微仿真QPG与预热critic训练自中心视觉四旋翼穿缝策略，并在真实实验验证。
核心要点
- 窄缝穿越需从高维视觉直接控制四旋翼，行为克隆或全展开BPTT易限制性能或训练成本高。
- 第一阶段用含缝隙几何的特权观测训练专家actor与critic，且无需沿优化参考轨迹重置。
- 第二阶段用两路自中心相机二值缝隙掩码和低维观测训练视觉策略，并预热特权critic。
- 相比冷启动critic或全展开BPTT，训练效率和穿越成功率更高，并能泛化到未见形状缝隙。
验证范围与局限：验证范围包括真实世界窄缝穿越，视觉输入为在线渲染的二值掩码。
英文摘要 (Abstract)
Traversing narrow gaps is challenging for autonomous quadrotors, especially when control commands come directly from high-dimensional visual observations. Existing end-to-end methods often rely on behavior cloning or full-rollout backpropagation through time (BPTT) via differentiable simulation, which can limit policy performance or incur high training costs. We propose a two-stage reinforcement learning framework for more efficient ego-centric visuomotor gap-traversal policy training, leveraging quasi-analytical policy gradients (QPG) via differentiable simulation and critic warm-starting. The framework utilizes QPG to avoid backpropagation through visual rendering, reducing computation and memory costs while improving sample efficiency. In the first stage, an expert actor and critic are trained using privileged observations, including gap geometry. Unlike prior gap-traversal approaches, our training utilizing QPG does not require resetting the agent along optimized reference trajectories. In the second stage, a visual policy is trained using binary gap masks from two ego-centric cameras and low-dimensional observations, while its privileged critic is warm-started from the first stage. This substantially improves training efficiency and traversal success compared with cold-starting the critic or using full-rollout BPTT. Our framework does not require retraining the expert actor when system parameters change, enabling more efficient generalization across drone platforms than state-of-the-art visual gap-traversal methods based on action supervision. The learned visual policy also generalizes to gaps with unseen shapes. Extensive real-world experiments further demonstrate robust gap traversal using binary masks rendered online. Beyond gap traversal, the proposed framework is generic and can be extended to other visuomotor robot learning tasks.
作者：Nuthasith Gerdpratoom, Tianchen Sun, Yichao Gao, Lin Zhao
来源
本页事实对应的公开页面，以原文为准。