具身开发者社区

外观

论文arXiv 2609.204771 个月前

几何变化下机器人插入的视觉 Sim-to-Real 学习：在钢筋安装中的应用 ​

针对几何变化下钢筋插入需反复采集真机数据的问题，RebarSim全仿真训练并在真机零样本验证。

AI 整理
  • 仿真与训练
  • 具身模型

原文链接：Visual Sim-to-Real Learning for Robotic Insertion under Geometric Variations: Application to Rebar Installation（arXiv 2609.20477 · 发布日期：2026-09-17）

Visual Sim-to-Real Learning for Robotic Insertion under Geometric Variations: Application to Rebar Installation

针对几何变化下钢筋插入需反复采集真机数据的问题，RebarSim全仿真训练并在真机零样本验证。

核心要点 ​

  • 钢筋插入间隙仅1.4毫米，设计与制造公差变化使真机数据需随批次重采。
  • 先用强化学习训练特权状态教师，再蒸馏为多视角RGB与本体输入的学生策略。
  • 学生策略零样本迁移到真实世界，在真实工厂钢筋的真机rollout中成功率91.3%。
  • 几何多样性、预训练、外观随机化和DAgger混合均被报告为关键；代码和资产已开放。

验证范围与局限：验证范围为钢筋安装插入任务，真机实验使用来自真实工厂生产批次的钢筋。

英文摘要 (Abstract) ​

Rebar insertion is among the most repetitive and physically demanding tasks on construction sites, and a contact-rich problem at 1.4 mm clearance. The parts, however, vary at two levels: a nominal design per structural member, and fabrication tolerance around each nominal design. Real-world data therefore has to be re-collected as designs and batches change. We present RebarSim, a visual sim-to-real system trained entirely in simulation. A privileged state-based teacher is trained with reinforcement learning over procedurally generated rebar geometries, then distilled into a multi-view student that maps raw RGB and proprioception directly to actions under extensive domain randomization. The student transfers to the real world zero-shot, seating rebars taken from a real factory production run in 91.3% of real-robot rollouts. Underlying that result, geometry diversity and pretraining both bring benefits. Training across a diverse set of nominal designs rather than one lifts the zero-shot success of both the teacher and the student on unseen designs, and the student policy outperforms a single-design specialist on that specialist's own design. A pretrained student then adapts to a new design with 4--6x fewer distillation samples than one trained from scratch. Visual sim-to-real transfer depends on appearance randomization and the DAgger mixture: removing either one sharply lowers success. Videos, code, and task assets are available at https://rebarsim.github.io.

作者：Tao Sun, Beining Han, Patrick Yin, Rui Xu, Harry He, Abhishek Gupta, Szymon M. Rusinkiewicz, Yi Shao

来源

本页事实对应的公开页面，以原文为准。

  1. arXiv 2609.20477：Visual Sim-to-Real Learning for Robotic Insertion under Geometric Variations: Application to Rebar Installation访问 2026-10-21