外观
论文arXiv 2610.1223120 天前
HT-Policies残差建模弥合回归与生成策略差距，推理更快且成功率持平
本文由 AI 根据公开内容整理，可能有误，请以原文为准：Residual Modeling Closes the Regression and Generative Policy Gap in Robot Learning（arXiv 2610.12231）
核心内容
针对机器人模仿学习中直接回归（MSE-Policies）性能常逊于生成式策略（Flow-Policies）的问题，研究从统计建模视角揭示根源：真实演示数据的动作预测残差呈现显著的状态依赖性与重尾分布。MSE损失对大残差样本分配过高梯度权重导致优化受损。为此提出的HT-Policies采用异方差Student-t回归，有效降低重尾干扰，在四项仿真基准及真机任务中实现与生成式基线持平的成功率，同时保持更快的训练与推理效率。
技术上，HT-Policies通过单次前向传播预测动作块（Action Chunks），并支持复用预训练的Flow-Matching、VLA及World-Action模型作为骨干网络。相比扩散或流匹配模型需多步采样，该架构保留了直接回归的计算高效性，同时解决了传统MSE对非高斯噪声敏感的工程痛点。实验表明，无论是从头训练还是基于预训练模型微调，HT-Policies均能在复杂操作任务中维持高鲁棒性，为具身智能策略学习提供了兼顾精度与实时性的新范式。
关键看点
- 残差重尾机制：分析证实真实机器人演示数据存在状态依赖的重尾残差，MSE因对异常值梯度过大导致优化偏差
- 异方差Student-t回归：HT-Policies学习输入依赖的残差尺度，单次前向传播输出动作块，抑制重尾噪声对梯度的干扰
- 兼容预训练骨干：支持直接复用Flow-Matching、VLA及World-Action等预训练模型权重，降低从头训练的数据与算力门槛
- 效能与精度对齐：在四项仿真及真机评测中成功率匹敌生成式策略，且训练收敛与推理延迟显著优于扩散/流匹配基线
工程与产业观察
该工作修正了“生成式策略必然优于回归”的工程认知，证明通过改进损失函数适配真实数据分布，直接回归即可达到SOTA水平。对于端侧部署资源受限或要求高频控制的具身系统，HT-Policies提供了比扩散模型更具工程落地价值的替代方案，尤其在复用大规模预训练模型时能显著缩短Sim-to-Real迭代周期。
英文摘要 (Abstract)
Learning from demonstration has enabled impressive robot behaviors. A common choice for policy learning is to use diffusion or flow matching (Flow-Policies), which often outperforms direct action regression trained with mean squared error (MSE-Policies). This gap is commonly attributed to multimodal demonstrations. We revisit this gap from the perspective of statistical modeling: how action-prediction residuals shape policy optimization. Our analysis of real-world robot demonstration data reveals substantial state-dependent variation in residual scales and heavier-than-Gaussian tails. While both MSE-Policies and Flow-Policies exhibit heavy-tailed action residuals, their training gradients behave differently: MSE allocates more gradient magnitude to observations with large action residuals, which hurts optimization. Motivated by these findings, we introduce heteroscedastic Student-t action regression (HT-Policies), which learns input-dependent residual scales and reduces the influence of heavy tails. HT-Policies predict action chunks with a single feed-forward pass and can reuse pretrained flow-matching-based policy networks as the backbone. Across four simulation benchmarks and real-robot evaluations, HT-Policies achieves success rates competitive with generative policy baselines, both when trained from scratch and from pretrained vision-language-action and world-action models, despite being faster in training and inference. Together, these findings shed light on the practical advantages of generative objectives in robot learning from demonstrations and offer an efficient direct-regression alternative for a range of architectures and tasks. Project page: https://projects.thelabone.org/regression-policy/
作者：Yuchen Zhou, Jiacheng You, Weikang Wan, Weijun Dong, Yang Gao, Jiayuan Mao
来源
本页事实对应的公开页面，以原文为准。