论文arXiv 2611.08097今天

STEP框架赋能Noetix E1人形机器人盲视下楼梯成功率达98%

提出STEP单阶段训练框架，通过双向掩码预测与动作条件化状态演化联合监督跨模态关系。在23自由度Noetix E1上实测，百次复杂地形试验成功98次，验证了显式预测机制在视觉降级下的鲁棒性。

AI 整理

仿真与训练
具身模型

本文由 AI 根据公开内容整理，可能有误，请以原文为准：STEP: Spatiotemporal Training with Explicit Prediction for Perceptive Humanoid Locomotion（arXiv 2611.08097）

核心内容

针对感知型人形机器人地形几何相关性依赖本体状态的问题，研究提出STEP（Spatiotemporal Training with Explicit Prediction）单阶段框架。该方法在共享控制表征中联合监督跨模态关系与动作依赖的状态演化，解决了传统特征聚合缺乏显式约束的痛点。在23自由度Noetix E1人形机器人实机部署中，涵盖楼梯、40cm平台、70cm间隙及踏脚石等场景的100次硬件试验成功98次，并在无后方视野条件下完成倒立下楼梯任务。

技术上，STEP利用多个可学习查询将深度图与本体感受token聚合为紧凑策略状态。双向掩码预测机制利用可见模态预测缺失模态的潜在特征以显式监督地形-本体关系；动作条件化预测器则将当前策略状态与执行动作映射至下一融合状态，目标直接在策略表征空间定义。仿真对比显示其地形穿越性能优于现有融合方案，深度丢失与延迟实验进一步证实了双重预测目标在视觉输入降级时的有效性，确保了板载传感条件下的稳定运行。

关键看点

STEP架构：单阶段框架联合监督跨模态关系与动作依赖状态演化，通过双向掩码预测显式约束地形与本体特征的交互学习
状态预测机制：动作条件化预测器将当前策略状态与执行动作映射至下一时刻，目标直接在策略表征空间内通过指数移动平均定义
实机验证数据：23-DoF Noetix E1搭载板载传感，在楼梯、40cm高台及70cm间隙等场景100次试验中成功98次
视觉鲁棒性：深度丢失与延迟实验表明，双重预测目标使机器人在视觉输入降级或完全盲视（如倒立下楼梯）时仍保持稳定

工程与产业观察

STEP证明了在人形机器人运动控制中，仅靠特征拼接不足以建立可靠的地形-本体关联，必须引入显式的时空预测约束。该框架将跨模态对齐与动力学预测统一在策略表征空间，显著提升了纯板载感知下的非结构化地形适应能力，为具身智能从仿真到真机的Sim-to-Real迁移提供了高鲁棒性的算法范式参考。

英文摘要 (Abstract)

For perceptive humanoid locomotion, the relevance of terrain geometry depends on the robot's body state and how both evolve under its actions. Existing work has advanced terrain-aware sensory fusion and cross-modal prediction, improving how visual and proprioceptive information is combined and related. However, feature aggregation alone does not specify which cross-modal relationships should be learned, while prediction between simultaneous observations does not explicitly constrain their action-dependent evolution. We present Spatiotemporal Training with Explicit Prediction (STEP), a one-stage framework that jointly supervises cross-modal relationships and action-dependent state evolution within a shared control representation. Multiple learned queries aggregate depth and proprioceptive tokens into a compact policy state. Bidirectional masked prediction uses each visible modality to predict latent features of the omitted modality, explicitly supervising terrain-body relationships. An action-conditioned predictor maps the current policy state and executed action to the next fused state, with an exponential-moving-average target defined directly in the policy representation space. Simulation comparisons show overall improvements in terrain traversal over evaluated fusion alternatives and ablations, while depth-dropout and delay experiments demonstrate the benefits of both predictive objectives under degraded visual input. Deployed on a 23-DoF Noetix E1 humanoid with onboard sensing, STEP succeeds in 98 of 100 hardware trials across stairs, a 40,cm platform, a 70,cm gap, stepping stones, and backward stair traversal without rearward terrain visibility.

作者：Youhao Lyu, Chenghao Sun, Zhenxuan Ding, Shuting Zhang, Ziyang Dong, Wenhao Cui