论文arXiv 2611.236283 天前
LoRN低秩噪声引导生成式机器人策略,视频调控成功率提升26.2%
本文由 AI 根据公开内容整理,可能有误,请以原文为准:LoRN: Low-Rank Noise Steering for Generative Robot Policies(arXiv 2611.23628)
核心内容
针对预训练生成式机器人策略优化,arXiv论文提出Low-Rank Noise Steering(LoRN)强化学习方法。该方法在不更新基础模型权重前提下,通过学习选择初始噪声方向来修正策略输出。研究发现在世界-动作模型中,动作生成对视频噪声的敏感度显著高于动作噪声,但现有噪声引导技术难以适配此类高维输入。LoRN通过引入小型引导Actor与范数控制构造,实现了跨三个世界-动作模型及十个模型-任务对的有效扩展。
技术上,LoRN利用固定低秩随机投影将Actor控制维度与全量噪声维度解耦,确保合成噪声保持高斯样本典型尺度,从而同时支持视频与动作输入的精准引导。量化评测表明,视频噪声引导使平均成功率较基础策略提升26.2个百分点,远超DSRL动作引导的7.1个百分点;在较小动作策略上亦能改善早期训练收敛性。关键验证显示,仅在仿真环境训练的引导Actor可直接零样本迁移至两个基础策略的三个真实物理双臂操作任务,无需额外实机训练。
关键看点
- 低秩投影解耦:采用固定低秩随机投影分离控制与噪声维度,结合范数控制构造,解决高维世界-动作模型的噪声引导扩展瓶颈
- 视频噪声敏感性:证实世界-动作模型中动作对视频噪声更敏感,视频引导成功率提升26.2%,显著优于DSRL动作引导的7.1%增幅
- Sim-to-Real零样本迁移:仿真训练的引导Actor无需真实数据微调,直接部署于两类基础策略的三个物理双臂操作任务并有效执行
- 免权重更新优化:作为即插即用推理时干预手段,仅通过调整初始噪声分布即可提升预训练生成式策略性能,避免昂贵的全参数微调
工程与产业观察
LoRN揭示了生成式具身策略中视频潜空间对行为生成的强杠杆效应,为World-Action Model的后训练优化提供了轻量级范式。其低秩解耦设计与Sim-to-Real零样本迁移能力,大幅降低了大模型适配新任务的算力与数据采集门槛,对解决通用机器人策略落地中的长尾场景泛化问题具有重要工程参考价值。
英文摘要 (Abstract)
Noise steering improves pretrained generative robot policies by learning to select their initial noise without updating model weights. We find that actions in the evaluated world-action models are more sensitive to video noise than action noise, making video noise an expressive input for steering. However, the noise-steering methods we evaluate, in their published forms, struggle to scale to the high-dimensional noise inputs of large policy classes such as world-action models. We introduce Low-Rank Noise Steering (LoRN), a reinforcement learning method that scales noise steering to world-action models. A small steering actor selects a direction through a fixed low-rank random projection, while our norm-controlled construction keeps the resulting noise near the typical scale of a Gaussian sample. This separates the actor's control dimension from the full noise dimension, enabling steering of both video and action inputs. Across three world-action models and ten model-task pairs, video steering improves average success by 26.2 percentage points over the base policy, compared with 7.1 points for DSRL action steering. For smaller action policies, the same construction achieves comparable final success while generally improving early-training success. Steering actors trained in simulation also transfer to three physical bimanual manipulation tasks across two base policies, without additional real-world training.
作者:Wei Liu, Abrar Anwar, Huihua Zhao, Soha Pouya, Yan Chang
外部来源
本页事实对应的公开页面,以原文为准。(点击将离开本站)