论文arXiv 2611.04255今天

ActOR框架融合环境AI与数字孪生提升手术室人形机器人自主性

研究提出ActOR框架，通过环境相机与人形机器人头部视觉构建共享数字孪生，由LLM智能体规划动作。仿真中探索调用降至0.12次/轮，扰动下成功率从8.3%升至51.7%；物理样机完成14/20次处置任务并验证重规划能力。

AI 整理

感知与导航
具身模型
评测与安全

本文由 AI 根据公开内容整理，可能有误，请以原文为准：ActOR: Ambient AI-Based Agentic Orchestration for Humanoid Autonomy in Operating Rooms（arXiv 2611.04255）

核心内容

针对手术室人形机器人因本体感知受限导致自主作业困难的问题，研究团队提出ActOR（Ambient AI-Based Agentic Orchestration）框架。该系统将墙面固定环境相机与机器人头部相机融合，构建物体级共享数字孪生（DT），并由大语言模型（LLM）智能体基于预编程运动与操作技能进行任务规划，无需针对特定任务训练。实验显示，在目标初始不可见场景下，环境感知使单轮探索调用从1.05次降至0.12次；引入闭环反馈后，标称执行成功率从46.7%提升至83.3%，在注入导航、物体位移及抓取扰动条件下，成功率从8.3%跃升至51.7%。

技术实现上，ActOR通过多视角视觉融合生成实时更新的数字孪生作为LLM规划器的全局状态表征，执行结果与DT更新反向驱动重试、替代策略或重规划决策。在基于真实手术室重建的两个仿真环境中验证了感知增强效果；在物理模拟手术室中，搭载该框架的人形机器人完成了20次单物体处置试验中的14次，并在场景动态变化后成功触发重规划。结果表明，环境AI提供的全局场景状态显著优于纯本体观测，结合技能执行反馈可有效支撑人形机器人在复杂医疗环境中的多步自主作业。

关键看点

共享数字孪生：融合墙面环境相机与机器人头部相机，构建物体级DT作为LLM规划器全局状态输入
探索效率提升：目标不可见时，环境感知使单轮Explore调用从1.05次降至0.12次，大幅减少无效搜索
鲁棒性增强：闭环反馈机制使扰动条件下任务成功率从8.3%提升至51.7%，支持动态重规划
零样本任务执行：LLM智能体基于预编程技能库规划，无需针对手术室处置任务进行额外训练

工程与产业观察

ActOR验证了“环境感知+数字孪生+LLM规划”架构在人形机器人复杂场景自主作业中的工程可行性。其核心价值在于通过外部基础设施补全本体感知盲区，并以结构化DT替代原始点云作为LLM上下文，显著降低探索开销与对端到端训练的依赖。该范式为医疗、仓储等半结构化环境中具身智能的快速部署提供了可复用的系统级参考路径。

英文摘要 (Abstract)

Operating rooms (ORs) increasingly contain ambient sensing infrastructure to enable AI-based optimization of workflows and utilization. We pursue an alternative path: using ambient AI to guide robot action and optimize utilization. Deploying humanoids in the OR without ambient AI is severely limited by cumbersome de novo orientation and planning due to partial onboard observations. We posit that this can be achieved by coupling an ambient AI-based agentic orchestrator with the humanoid robotic actor, providing shared perception and execution feedback for humanoid autonomy in the OR. ActOR fuses a wall-mounted ambient camera and the robot's head-mounted camera into a shared object-level digital twin (DT). A large language model agent plans over the DT using pre-programmed locomotion and manipulation skills without task-specific training. During execution, DT updates and execution outcomes inform retries, alternative approaches, or replanning. We evaluated ActOR in simulated and physical environments. Across three tasks in two simulated ORs reconstructed from real rooms, ambient perception reduced exploration from 1.05 to 0.12 Explore calls per trial when targets were initially outside the robot's view. Closed-loop feedback increased success from 46.7% to 83.3% under nominal execution and from 8.3% to 51.7% under injected navigation, object-relocation, and grasp perturbations. In a physical mock OR with a humanoid, ActOR completed 14 of 20 single-object disposal trials and demonstrated replanning after scene changes. Ambient AI-based observations provide scene state that improves robot action planning, increasing success and reducing exploration compared with robot-only observations. Feeding skill outcomes and DT updates back to the planner improves multi-step task success. Our results show the feasibility of humanoid task autonomy for multi-step disposal tasks in a mock OR.

作者：Han Zhang, Sicheng Lu, Aleks Santari, Jeffrey K. Jopling, Masaru Ishii, Mathias Unberath