论文arXiv 2611.167529 天前
AgriSense框架:虚拟温室四足机器人多模态决策与干预验证
本文由 AI 根据公开内容整理,可能有误,请以原文为准:AgriSense: Vision-Language Decision Support and Robotic Intervention in a Virtual Greenhouse Copy title(arXiv 2611.16752)
核心内容
针对自主温室管理需求,研究提出AgriSense框架,整合Isaac Sim虚拟环境、四足移动操作机器人及RGB/LiDAR惯性传感系统。该框架采用冻结DINOv2与BERT编码器联合预测12项环境参考值、目标区域及干预标志。测试表明,Checkpoint重放复现了43次区域决策中的41次;在5个随机种子的源分组测试集中,完整模型、去图像及去语言配置均实现44次中43次的正确区域决策,证实当前架构下多模态融合未带来区域精度的增量提升。
完整模型在温度、湿度和土壤湿度的平均绝对误差分别为0.249℃、2.030%和0.997%。纯传感器输入的区域决策精度与全模型持平,而纯语言输入仅为68.64%,且外部真实图像基准暴露出分布偏移局限。导航方面,5条离线Point-LIO路线的刚性对齐轨迹误差为5.22-17.32mm;40个虚拟臂目标中32个满足几何验收标准。目前系统集成依赖仿真器状态输入,物理温室性能及农艺处理有效性尚未完成实地验证。
关键看点
- 多模态消融:完整、去图像及去语言模型区域决策准确率均为43/44,未见多模态融合对区域判断的增量收益
- 环境预测精度:全模型温度MAE为0.249℃,湿度2.030%,土壤湿度0.997%,纯传感器输入可匹配全模型区域精度
- 导航与控制:离线Point-LIO轨迹误差5.22-17.32mm,40个虚拟机械臂目标中32个通过几何验收,8个边界拒绝
- 泛化局限:外部真实图像基准测试揭示显著分布偏移问题,物理温室环境下的农艺干预 efficacy 尚未验证
工程与产业观察
该工作揭示了具身智能在结构化农业场景中的关键工程瓶颈:在特定任务下,昂贵的VLM多模态融合未必优于纯感知方案,且Sim-to-Real存在显著分布偏移。对于农业机器人研发,应优先评估单模态基线性能,并警惕仿真器状态依赖导致的虚实迁移失效风险,需补充真实农艺数据闭环验证。
英文摘要 (Abstract)
Autonomous greenhouse management increasingly requires robotic systems that integrate perception, navigation, environmental regulation, and plant-level intervention. This paper presents AgriSense, a framework combining an Isaac Sim virtual greenhouse, a quadruped mobile manipulator, RGB and LiDAR-inertial sensing, environmental control, and supervised vision-language decision support. Frozen DINOv2 and BERT encoders, together with structured environmental inputs, support joint prediction of twelve environmental references, a target zone, and an intervention flag. Checkpoint replay reproduces 41/43 zone decisions. Across five seeds on a source-grouped test set, the full, image-removed, and language-removed configurations each achieve 43/44 correct zone decisions, providing no evidence of incremental multimodal zone-accuracy improvement. The full model achieves mean absolute errors of 0.249 degrees C for temperature, 2.030 percentage points for humidity, and 0.997 percentage points for soil moisture. Sensor-only inputs match full-model zone accuracy, while language-only inputs achieve 68.64%. External real-image benchmarks reveal distribution-shift limitations. Five offline Point-LIO routes yield rigidly aligned trajectory errors of 5.22-17.32 mm. Of 40 virtual-arm targets, 32 satisfy geometric acceptance criteria and eight are rejected at boundaries. Six long-horizon climate cases and fault protocols characterize integration limits. Navigation and integrated treatment rely on simulator-state inputs; physical greenhouse performance and agronomic treatment efficacy remain unvalidated. The contribution is an evaluated decision-support and robotic-intervention framework for a virtual greenhouse.
作者:Lyes Saad Saoud, Oualid Doukhi, Reza Ghorbani, Moussa Ayyash
外部来源
本页事实对应的公开页面,以原文为准。(点击将离开本站)