外观
论文arXiv 2610.2166613 天前
微分同胚隐藏状态引导：通过专家隐藏状态几何提升模仿学习策略鲁棒性
原文链接：Diffeomorphic Hidden-State Steering: Improving Imitation Learning Policy Robustness through Expert Hidden-State Geometry（arXiv 2610.21666 · 发布日期：2026-10-15）
Diffeomorphic Hidden-State Steering: Improving Imitation Learning Policy Robustness through Expert Hidden-State Geometry
针对闭环执行中误差累积问题，DHS利用专家隐藏状态几何在测试时引导，并在四个仿真操作任务验证提升抗观测噪声成功率。
核心要点
- 闭环执行中微小观测或动作误差会累积，破坏已训练的模仿学习策略。
- DHS从冻结策略回放专家演示得到隐藏状态，学习可逆非线性坐标映射。
- 方法更强衰减低专家方差坐标，再映回隐藏状态，使引导方向随状态变化。
- 四个仿真操作任务中，相比未修改Transformer策略提升观测噪声下平均成功率。
验证范围与局限：验证范围为四个仿真操作任务的观测噪声设置；与动作平滑结合只在部分任务中报告改进。
英文摘要 (Abstract)
Small observation or action errors can compound during closed-loop execution and disrupt a trained imitation learning policy. We study whether the original expert demonstrations can improve robustness through test-time hidden-state steering without additional demonstrations or policy retraining. Our analysis of a trained policy finds that expert hidden-state variation is concentrated in relatively few dimensions, while the directions capturing this variation differ across regions. Motivated by this geometry, we propose Diffeomorphic Hidden-State Steering (DHS), which learns an invertible nonlinear coordinate map from expert hidden states obtained by replaying the demonstrations through the frozen policy. DHS attenuates coordinates with low expert variance more strongly than those with high variance, then maps the modified coordinates back to hidden-state space. This allows steering directions to vary with the current hidden state while retaining every coordinate. Across four simulated manipulation tasks, DHS improves mean success under observation noise over the unmodified Transformer policies. Controlled comparisons yield higher mean success with learned nonlinear coordinates than with linear coordinates under both hard projection and full-rank rescaling. Combining DHS with action smoothing improves mean success rate under observation noise in certain tasks compared with action smoothing alone.
作者：K. M. Lee
来源
本页事实对应的公开页面，以原文为准。