外观
论文arXiv 2606.000905 个月前
Physical AI静默失效综述：构建运行时动作授权安全边界
本文由 AI 根据公开内容整理，可能有误，请以原文为准：Silent Failures in Physical AI: A Literature Review of Runtime Action Authorization for Autonomous Systems（arXiv 2606.00090）
核心内容
随着机器人基础模型、VLA及世界模型将多模态观测映射为物理动作，传统内容审核与经典机器人安全已无法覆盖新型风险。arXiv 2606.00090 综述指出，黑盒模型可能在传感器漂移、遮挡或分布偏移下输出语义合理但物理违规的指令，导致下游控制器检测前的“静默失效”。现有技术在模型能力与安全机制上各自演进，缺乏统一的运行时授权边界来阻断此类风险。
该文建立了有界问题公式与静默物理动作失效定义，系统梳理了跨具身基础模型、仿真、安全基准及运行时保障的技术缺口。核心贡献在于提出了运行时护栏功能分类法及评估要求，旨在将不确定性估计、验证与Guardrail评价整合为完整的Physical AI保障机制，为算法工程师在黑盒模型与硬件执行层之间构建可验证的安全拦截器提供理论框架与评测标准。
关键看点
- 静默失效定义：指模型在传感器漂移、状态估计错误或幻觉 affordance 下，输出高置信度语义对齐但物理违规的动作，且未被下游控制器即时拦截
- 安全边界缺失：综述确认当前具身模型、世界模型与安全控制研究各自独立，尚无单一技术流能提供黑盒模型到物理执行的完整运行时授权边界
- 护栏功能分类：构建了运行时 Guardrail 功能分类体系与评估要求，将不确定性估计、形式化验证与安全基准整合为 Physical AI assurance 机制
工程与产业观察
该综述揭示了具身智能从“语义对齐”迈向“物理安全”的关键工程缺口。对于部署VLA或世界模型的团队，仅靠模型端对齐不足以防范静默失效，必须在推理与执行间插入独立的运行时授权层。文中提出的护栏分类与评估标准，为构建可量化验证的具身安全中间件提供了直接参考，是解决端到端模型不可解释性风险的必要工程路径。
英文摘要 (Abstract)
Physical AI systems increasingly map multimodal observations, language instructions, and learned world representations into physically consequential actions. Robotics foundation models, vision-language-action models, and world-model-based autonomous systems can condition decisions that move vehicles, robots, drones, and industrial machines. This transition exposes a safety problem that is not fully captured by conventional AI content moderation or by classical robot safety alone: a black-box model may issue a physically consequential action while appearing confident, plausible, and semantically aligned. The resulting failure can be silent, arising from sensor drift, occlusion, state-estimation error, distribution shift, hallucinated affordances, or invalid physical assumptions before downstream hardware controllers detect a violation. Across embodied foundation models, world models, robotics simulation, embodied safety benchmarks, safe control, runtime assurance, uncertainty estimation, verification, and guardrail evaluation, model capability and safety mechanisms have advanced along largely separate technical tracks. A recurring gap synthesized here is that no single stream surveyed in this review supplies a complete runtime authorization boundary between black-box Physical AI models and physical execution. The resulting analysis develops a bounded problem formulation, a definition of silent physical-action failure, a taxonomy of runtime guardrail functions, and evaluation requirements for comparing guardrails as Physical AI assurance mechanisms.
作者：Barak Or
来源
本页事实对应的公开页面，以原文为准。