外观
论文arXiv 2602.063823 个月前
现在你看到了：从原始像素学习端到端人形机器人运动
原文链接：Now You See That: Learning End-to-End Humanoid Locomotion from Raw Pixels（arXiv 2602.06382 · 发布日期：2026-07-13）
Now You See That: Learning End-to-End Humanoid Locomotion from Raw Pixels
针对视觉人形运动的感知噪声与多地形训练冲突，提出端到端框架，并在两个人形平台上验证。
核心要点
- 视觉人形运动受 sim-to-real 感知噪声影响，细粒度任务性能易下降。
- 方法构建高保真深度传感器仿真，模拟立体匹配伪影和标定不确定性。
- 通过视觉感知行为蒸馏，将特权高度图知识迁移到带噪深度观测。
- 实验显示策略可在多样环境中稳健运行，处理高平台、宽间隙和双向长期楼梯任务。
验证范围与局限：验证范围为两个人形平台、不同立体深度相机及多样环境中的运动任务。
英文摘要 (Abstract)
Achieving robust vision-based humanoid locomotion remains challenging due to two fundamental issues: the sim-toreal gap introduces significant perception noise that degrades performance on fine-grained tasks, and training a unified policy across diverse terrains is hindered by conflicting learning objectives.To address these challenges, we present an end-to-end framework for vision-driven humanoid locomotion.For robust sim-to-real transfer, we develop a high-fidelity depth sensor simulation that captures stereo matching artifacts and calibration uncertainties inherent in real-world sensing.We further propose a vision-aware behavior distillation approach that combines latent space alignment with noise-invariant auxiliary tasks, enabling effective knowledge transfer from privileged height maps to noisy depth observations.For versatile terrain adaptation, we introduce terrain-specific reward shaping integrated with multi-critic and multi-discriminator learning, where dedicated networks capture the distinct dynamics and motion priors of each terrain type.We validate our approach on two humanoid platforms equipped with different stereo depth cameras.The resulting policy demonstrates robust performance across diverse environments, seamlessly handling extreme challenges such as high platforms and wide gaps, as well as fine-grained tasks including bidirectional long-term staircase traversal.
作者：Wandong Sun, Yongbo Su, Leoric Huang, Zilong Zhang, Dwyane Wei, Sen Xu, Sheng Cao, Xingwu Tian, Baoshi Cao, Yang Liu
来源
本页事实对应的公开页面，以原文为准。