外观
论文HF Daily Papers · arXiv 2603.192338 天前
VLA模型机制研究揭示pi0.5动作专家绑定绝对坐标致泛化失效
研究通过可解释性管线发现pi0.5等VLA模型动作专家绑定绝对坐标，导致场景微扰即失败。在UR10e实机验证中，激活注入使99.6%回合执行错误轨迹，目标位移10-15cm后抓取完全失效，开源Action Atlas平台含520+个SAE与79个概念。
Universal Robots UR10e本文由 AI 根据公开内容整理，可能有误，请以原文为准：Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models（HF Daily Papers · arXiv 2603.19233）
核心内容
针对Vision-Language-Action模型在微调后对场景扰动极度敏感的问题，研究团队开发了机械可解释性管线。实验显示，将任务A的激活值注入正在观察任务B场景的pi0.5策略动作专家中，99.6%的回合（n=1,968）会强制执行任务A的运动轨迹。该现象证实模型内部表征与绝对工作空间坐标强绑定而忽略视觉输入，从机制上解释了现有基准测试中报告的扰动脆弱性根源。
在Universal Robots UR10e物理平台上，定向重放成功在26/26次单向rollout中重定向任务；当源物体移动10-15cm时，机械臂仍在记录坐标执行抓放，50/50次试验均失败。基于LIBERO、MetaWorld等环境42万+次rollout分析发现，所有架构均编码指令但权重低于视觉，且目标选择与运动生成在VLM与专家路径中分离。团队已开源Action Atlas平台，包含520+个训练好的SAE、79个识别概念及UR10e评分数据。
关键看点
- 绝对坐标绑定：pi0.5动作专家激活注入实验显示99.6%回合无视当前视觉场景，强制复现源任务运动轨迹，证实模型缺乏相对空间泛化能力
- UR10e实机验证：物理平台定向重放成功率达100%，目标物体位移10-15cm后抓取完全失效，量化了VLA模型在真实硬件上的坐标依赖程度
- 架构解耦机制：跨6种VLA架构42万+次rollout表明，目标选择与电机生成在VLM和专家路径中分离，且指令权重普遍低于视觉特征
- 开源可解释工具：发布Action Atlas平台集成520+个SAE稀疏自编码器与79个语义概念，提供UR10e实机rollout评分数据支持社区复现研究
工程与产业观察
该研究从神经元激活层面证实主流VLA模型存在绝对坐标过拟合缺陷，揭示了Sim-to-Real泛化失败的内在机制。对于具身智能算法工程师，这提示在策略训练中必须引入显式相对坐标表征或数据增强以解耦空间记忆；Action Atlas提供的SAE探针与概念图谱为VLA模型调试与安全性评估提供了标准化的工程化工具链。
英文摘要 (Abstract)
A fine-tuned Vision-Language-Action (VLA) policy can easily place the alphabet soup in the basket on command, just as it saw during fine-tuning. However, if the basket is shifted five centimeters to the left, it drops the soup off the table. We developed an interpretability pipeline to investigate what such a policy is computing. Injecting task A's activations into the action expert while the policy looks at task B's scene makes pi0.5 execute task A's motor trajectory in 99.6% of episodes ($n{=}1{,}968$). The injected program is bound to absolute workspace coordinates and ignores the visible scene, which provides a mechanistic account of the perturbation brittleness that concurrent benchmarks report. The binding holds on a physical UR10e: directed replay redirects the commanded task in 26/26 rollouts in one direction and 18/26 in the other, with no-injection and random-direction controls at the floor, and once the source object is moved 10--15,cm the arm grasps and releases at the recorded coordinates in 50/50 trials, picking up the wrong block when one sits there and closing on empty air when none does. Applying the same interventions to six VLAs on LIBERO, MetaWorld, SimplerEnv, and ALOHA over 420{,}000+ rollouts yields three further findings: every architecture encodes the instruction, and four of the five weight it below vision whenever the scene alone identifies the goal; under the standard replacement write-back the SAE pooling that preserves behavior is architecture-specific (pi0.5 per-token, SmolVLA indifferent), and goal selection and motor generation separate into VLM and expert pathways wherever the two are distinguishable. We release \textbf{Action Atlas} (https://action-atlas.com), an interactive platform covering all 520+ trained SAEs, 79 identified concepts, and the scored UR10e rollouts. Code: https://github.com/CWRU-AISM/action-atlas
Hugging Face Daily Papers 热度：1 upvotes 作者：Bryce Grant, Xijia Zhao, Peng Wang
来源
本页事实对应的公开页面，以原文为准。