论文arXiv 2611.0889513 天前
OpenVLA安全指令遵循失效:79%违禁操作率与Layer12前干预机制
本文由 AI 根据公开内容整理,可能有误,请以原文为准:Localizing and Editing the Goal Representation of a Vision-Language-Action Model(arXiv 2611.08895)
核心内容
针对Vision-Language-Action模型OpenVLA的安全对齐问题,研究团队在LIBERO-Goal基准上测试了其对“禁止将碗放在炉子上”等否定指令的遵循能力。结果显示模型存在严重的安全对齐缺陷,在包含否定词的rollout中,79%的情况下仍执行了被禁止的任务,与普通指令78%的完成率几乎无异。线性探针分析表明,模型内部对“not X”与“X”的目标表征读数高度相似,证实单纯依赖自然语言否定词无法有效触发VLA模型的安全规避机制。
激活补丁与转向实验揭示了OpenVLA目标表征的层级分布规律:首动作主要依赖Layer 12前的指令Token及Layer 16后的末尾Prompt Token。仅在Layer 8应用固定转向向量可重定向18次中的12次rollout,而Layer 12至20的固定向量及Prefill补丁对158次尝试均无效;从Layer 16起,唯有在每个解码动作Token处打补丁才能生效(Layer 24成功率15/18)。此外,Layer 32的目标读出探针在检测运行时指令变更导致的违规方面,性能优于传统失败检测器,验证了外部目标监控作为安全兜底的有效性。
关键看点
- 安全对齐失效:OpenVLA在LIBERO-Goal上对否定指令缺乏敏感性,79%的违禁rollout被执行,目标探针显示正负指令内部表征无显著差异
- 关键干预窗口:目标表征在Layer 12前固化,仅Layer 8转向向量有效(12/18),Layer 12-20的固定向量与Prefill补丁完全失效
- 运行时修正代价:Layer 16后修改目标需对每个解码动作Token进行补丁,Layer 24逐Token补丁成功率为15/18,计算开销显著增加
- 外部监控优势:Layer 32目标读出探针检测指令变更违规的性能优于失败检测器,支持将安全约束移出指令流并实施独立监控
工程与产业观察
该研究为VLA模型安全部署提供关键工程坐标:否定式Prompt不可靠,安全约束必须外置为独立监控模块。算法层面,模型编辑需在Layer 12前完成或承担逐Token推理开销;这提示具身智能研发应构建独立于生成链路的安全校验层,而非过度依赖端到端模型的语义理解能力来规避物理风险。
英文摘要 (Abstract)
Operators can state safety constraints for robots as prohibitions such as "do not put the bowl on the stove". We study whether OpenVLA follows such prohibitions and where it represents the goal it executes. The 10 tasks of LIBEROGoal share one scene, so we can label the goal that each rollout executes. We combine closed-loop rollouts with activation patching, linear probes, activation steering and runtime monitors. On LIBERO-Goal, negation had no detectable effect on which task OpenVLA executed. The model executed the forbidden task in 79% of negated rollouts and completed 78% of plain tasks. A goal probe gives similar read-outs under "not X" and "X". Activation patching shows that the first action depends mostly on the instruction tokens before layer 12 and on the last prompt token from layer 16 onward. A fixed steering vector at the instruction tokens at layer 8 redirected 12 of 18 rollouts. Fixed vectors and prefill patches at layers 12 to 20 redirected none of 158. From layer 16, only patches at every decoded action token redirected the robot (15 of 18 rollouts at layer 24). When the instruction changes during a rollout, the layer-32 goal read-out detects non-compliance better than failure detectors, and an external probe on image and instruction features performs comparably. These results support enforcing prohibitions outside the instruction and monitoring goal read-outs against the specification. Goal edits in OpenVLA should act before layer 12 or patch every action token.
作者:Sudhanshu Srivastava, Faisal Aqlan
外部来源
本页事实对应的公开页面,以原文为准。(点击将离开本站)