论文arXiv 2611.07272今天

VREgo：消费级VR头显采集示教数据训练机器人操作策略

研究提出VREgo框架，利用PICO 4 Ultra等消费级VR头显采集第一人称示教数据训练机械臂操作策略。在立方体放置与堆叠任务仿真评测中，经轨迹适配与规则优化后，系统双阶段任务成功率达93.8%，验证了低成本VR设备替代专用采集设备的可行性。

AI 整理

仿真与训练
数据与采集

本文由 AI 根据公开内容整理，可能有误，请以原文为准：VREgo: Consumer VR Headsets for Learning Robot Manipulation from Egocentric Demonstrations（arXiv 2611.07272）

核心内容

arXiv论文2611.07272发布VREgo系统，旨在解决机器人模仿学习中高质量数据采集成本高昂的问题。该系统基于消费级VR头显PICO 4 Ultra构建EgoKit工具包，支持操作员在桌边直接进行第一人称示教。研究对比了PICO 4 Ultra与Meta Project Aria在相同41种布局下的数据采集效果，在40条示教数据训练下，PICO采集数据的策略在立方体放置任务中平均成功率约67%，虽略低于Aria的75%但显著优于其零成功种子，证实了消费级设备的有效性。

技术层面，VREgo改进了HumanEgo预处理流程，通过动态估计抓取阈值、释放后位置更新及关键帧掩码传播提升数据质量。针对开环夹爪位移物体及重规划延迟等问题，团队引入基于仿真器物体位置的执行规则并适配训练轨迹。优化后的系统在两阶段堆叠任务中，跨四个噪声种子的评估成功率从基线水平提升至93.8%。该项目代码与数据集已在vrego.chuange.org开源，为具身智能数据采集提供了低成本范式。

关键看点

低成本采集验证：基于PICO 4 Ultra头显采集40条示教数据，立方体放置任务仿真成功率达67%-74%，性能接近Project Aria专用设备。
数据预处理优化：改进HumanEgo算法，实现逐条记录抓取阈值估计、释放后物体位置更新及关键帧分割掩码自动传播。
执行策略增强：针对夹爪误触与重规划超时问题，结合仿真器真值添加任务特定执行规则，使双阶段堆叠成功率升至93.8%。
开源资产释放：完整框架VREgo及EgoKit工具包已在vrego.chuange.org开源，支持消费级VR头显快速部署机器人示教采集。

工程与产业观察

该工作证明了消费级VR头显在具身智能数据采集中的工程可用性，通过算法补偿弥补了硬件精度差距。对于缺乏昂贵遥操作设备的团队，VREgo提供了一套可复现的低成本Sim-to-Real数据管线，其针对抓取失败模式的规则修正思路对提升模仿学习鲁棒性具有直接参考价值。

英文摘要 (Abstract)

VREgo trains manipulation policies from demonstrations recorded on consumer VR headsets; we instantiate it with EgoKit on one such headset, the PICO 4 Ultra. On-device head and hand tracking and stereo object localisation allow the operator to begin a demonstration at the table. We adapt HumanEgo's preprocessing to estimate grasp thresholds per recording, update object positions after release, and propagate segmentation masks between keyframes. We evaluate policies in simulation on two tabletop tasks recorded by one operator. For cube placement, we compare PICO and Project Aria recordings of the same 41 layouts. With 40 training demonstrations, the PICO policies placed the cube on the tray in 67, 74, and 60 of 100 evaluation layouts across three training seeds. The corresponding Aria results were 73, 78, and 0. For a two-stage stacking task recorded with the PICO alone, the baseline placed the block on the tray in 94 of 100 layouts and met the end-of-episode criterion for both placements in two. In one diagnostic episode, repeated replanning deferred release beyond the eight executed steps. Demonstration replays also showed that the open gripper could displace the cube before grasping it. After adapting the training trajectories and adding task-specific execution rules that use simulator object positions, the system met both placement criteria in 93.8% of episodes across four evaluation noise seeds. The same 100 layouts informed rule development and evaluation. VREgo is available at https://vrego.chuange.org/.

作者：Liuchuan Yu