论文arXiv 2611.175666 天前
VLM驱动3D风险场构建,实现机器人操作任务级语义安全推理
本文由 AI 根据公开内容整理,可能有误,请以原文为准:Grounding Object-to-Object Hazard Reasoning into 3D Risk Fields for Task-Conditioned Safety(arXiv 2611.17566)
核心内容
针对机器人操作安全不仅依赖几何约束更需语义理解的问题,研究团队提出一种基于视觉语言模型(VLM)的任务条件安全推理框架。该系统利用结构化安全工具处理RGB-D观测与自然语言指令,首先判断任务是否因不可恢复风险而应被拒绝;若可执行,则识别可预防的物体对危害并指定需避免的空间关系,为操作安全提供语义层面的前置保障。
技术实现上,框架结合开放词汇分割与深度信息将检测对象锚定至3D空间,构建表征危险区域的连续风险场。该风险场被集成至避障模块,在执行阶段动态修正标称机器人运动以规避语义风险。实验覆盖仿真与真实环境,针对可预防危害、需拒绝任务及安全变体进行了系统评估,验证了该方法在处理复杂场景交互时的有效性与鲁棒性。
关键看点
- 语义安全推理:VLM基于RGB-D与指令判断任务可执行性,识别物体对危害并定义需规避的空间关系
- 3D风险场构建:融合开放词汇分割与深度数据,将语义危害映射为连续3D风险场表征危险区域
- 动态轨迹修正:风险场集成至避障模块,在执行阶段实时修正机器人标称运动以规避语义风险
- 多维安全验证:在仿真与真机环境中测试可预防危害、任务拒绝及安全变体,验证框架有效性
工程与产业观察
该工作将具身安全从纯几何避障拓展至语义风险建模,解决了VLM推理结果难以直接用于底层控制的痛点。通过构建连续3D风险场作为中间表征,实现了高层语义理解与低层运动规划的闭环耦合,为非结构化环境下机器人操作的安全对齐提供了可落地的工程范式。
英文摘要 (Abstract)
Safety in robotics does not solely depend on geometric constraints, but also on semantics: the given task, environment properties, and interactions between objects in the scene. We present a framework based on vision-language models (VLMs) to infer task- and scene-dependent safety constraints for robot manipulation. We provide the VLM a structured safety tool based on an RGB-D observation and a natural-language task instruction that first decides whether the task should be rejected since its execution may result in a non-recoverable hazardous outcome. Otherwise, it identifies preventable object-pair hazards and specifies the spatial relations that must be avoided. Open-vocabulary segmentation and depth information are then used to ground the detected objects in 3D and construct continuous risk fields that represent the corresponding hazardous regions. These fields are integrated with an obstacle avoidance module that corrects nominal robot motions during execution while accounting for the risk field. We evaluate the framework on preventable hazards, tasks requiring rejection, and safe variants in simulation and the real-world. Project website: https://vlm-hazard-riskfield.github.io
作者:Mariana Sosa Guzm'an, Yide Song, Amin Abyaneh, Gregory Dudek, Hsiu-Chin Lin
外部来源
本页事实对应的公开页面,以原文为准。(点击将离开本站)