外观
论文arXiv 2610.2128213 天前
LICOA：用于移动空间增强现实的指令条件光学动作学习
原文链接：LICOA: Learning Instruction-Conditioned Optical Actions for Mobile Spatial Augmented Reality（arXiv 2610.21282 · 发布日期：2026-10-15）
LICOA: Learning Instruction-Conditioned Optical Actions for Mobile Spatial Augmented Reality
LICOA针对SAR中位姿、区域或路径预设问题，用移动底盘、2DoF镜和多相机接口学习指令条件光学动作，并在未见真实环境闭环任务验证。
核心要点
- 多数SAR系统预设投影仪位姿、目标区域或运动路径，难以自主决定投影动作。
- LICOA用差速底盘实现房间级移动，2DoF可转镜和投影尺度控制实现局部光束调节。
- 共享接口融合外部和机载RGB相机与控制状态，将预训练动作模型映射为物理光学动作。
- 构建150个任务、7,500个渲染回合的仿真语料；最佳适配模型真机成功率达57.50%。
验证范围与局限：验证限于仿真语料训练后，在未见真实环境的12个任务、480次闭环试验和150次消融试验。
英文摘要 (Abstract)
Spatial augmented reality (SAR) augments physical environments with projected content, but most systems assume that the projector pose, target region, or motion path has been fixed or specified. We instead make projected light an embodied action: given a language instruction and current visual observations, a system must decide how to move and where and at what scale to place the projection. We introduce LICOA to realize this capability. Its differential-drive chassis provides room-scale approach and repositioning, while a steerable mirror with two degrees of freedom (2DoF) and projection scale provide rapid local control of the beam. A shared interface maps synchronized external and onboard RGB cameras with robot control state to physical optical actions across pretrained action models. Because instruction-conditioned optical action lacks dedicated training data and a common evaluation protocol, we construct a matched simulation embodiment and a corpus with 150 tasks and 7,500 rendered episodes, then evaluate models trained only on this corpus in previously unseen real-world environments. Across 480 complete closed-loop real-world execution trials (rollouts) on 12 tasks, the best adapted action model achieves a 57.50% success rate and 40.64% success weighted by path length, compared with 8.33% and 5.15% for the same architecture trained from scratch. A further 150 real-world ablation rollouts support the need for both camera views and coordinated control of the chassis and mirror. These results establish a physical platform, shared perception and optical action interface, corpus, and real-world evaluation protocol for treating projected light as an embodied action.
作者：Yuchen Deng, 周顺恺, Ruixuan Zhu, Haibin Ling, Bingyao Huang
来源
本页事实对应的公开页面，以原文为准。