Skip to content

论文arXiv 2611.149819 天前

单RGB图驱动宇树G1交互:两阶段优化实现70%物理执行成功率 ​

本文由 AI 根据公开内容整理,可能有误,请以原文为准:From a Single RGB Image to Robot Execution: Metric Scene Alignment and Contact-Guided Retargeting(arXiv 2611.14981)

核心内容 ​

针对人形机器人在新环境中交互依赖预建3D资产的痛点,研究提出一种从单张RGB图像直接生成可执行运动参考的统一管线。该方法集成场景几何重建、条件化人类交互生成及机器人运动重定向,并在仿真中取得89%成功率。在Unitree G1上进行的50次物理试验中,系统实现了70%的执行成功率,验证了无需针对特定场景微调策略即可跨环境部署的有效性。

为保障全流程几何一致性,管线引入双重优化机制:重定向前采用联合度量交互细化(JMIR)对齐人类动作与度量场景;重定向后利用接触引导残差重定向(CGRR)修正因具身差异导致的支撑误差。该架构使生成的运动参考能被多个预训练跟踪器直接执行。相比GRAIL方法,该管线将细化时间从480秒压缩至87秒,提速5.5倍,显著提升了单目视觉驱动具身智能的工程可用性。

关键看点 ​

  • 双重几何优化:采用JMIR对齐度量场景与CGRR修正支撑误差,确保单RGB图生成的动作在重定向后保持物理一致性
  • 实机验证指标:在Unitree G1上完成50次物理试验,无需场景特定策略微调,端到端执行成功率达到70%
  • 推理效率提升:相比GRAIL基准方法,运动细化耗时从480秒降至87秒,实现5.5倍加速,满足实时部署需求
  • 泛化执行能力:生成的运动参考兼容多种预训练人形跟踪器,解耦了场景理解与底层控制策略的强绑定关系

工程与产业观察 ​

该工作验证了单目视觉驱动具身交互的工程可行性,通过显式几何约束解决了生成式动作在物理世界的落地难题。87秒的优化耗时与70%的实机成功率表明,基于RGB的场景理解正从纯感知迈向可执行的机器人任务规划,为缺乏高精地图的非结构化环境作业提供了低成本技术路径。

英文摘要 (Abstract) ​

Extending humanoid interaction generation to new environments remains limited by the need for prebuilt 3D scene assets. Reconstructing scenes directly from single RGB images offers a way to overcome this limitation, but introduces a key challenge in maintaining geometric consistency throughout motion generation and transfer. Visually plausible human motions may be misaligned with the reconstructed scene, while retargeting can introduce additional support errors due to differences between human and robot embodiments. We present a unified pipeline that transforms a single RGB image into a humanoid motion reference. It reconstructs scene geometry from the input image, generates a human interaction conditioned on the reconstructed scene, and retargets the motion to a humanoid robot. To maintain geometric consistency throughout this process, we incorporate two complementary refinement stages before and after embodiment transfer. Before retargeting, Joint Metric Interaction Refinement (JMIR) refines human motion alignment with the registered metric scene while preserving the generated interaction structure. Following retargeting, Contact Guided Residual Retargeting (CGRR) corrects robot specific support errors to restore the intended support relationships. Together, these stages produce motion references that are consistent with the reconstructed scene and can be followed by multiple pretrained humanoid trackers without scene specific policy fine tuning. Compared with GRAIL, our method reduces refinement time from 480 s to 87 s, yielding a speedup of 5.5 times. The complete pipeline achieves a success rate of 89% in simulation and 70% across 50 physical trials on the Unitree G1, demonstrating its effectiveness in generating executable humanoid motion references from single RGB images.

作者:Zhenxuan Ding, Yu Huang, Xinmiao Wang, Chenghao Sun, Ziyang Dong, Wenhao Cui

基于本页技术方向关联的整机参数、核心零部件、实测报告与同赛道选型横评。

外部来源

本页事实对应的公开页面,以原文为准。(点击将离开本站)

  1. arXiv 2611.14981:From a Single RGB Image to Robot Execution: Metric Scene Alignment and Contact-Guided Retargeting ↗访问 2026-11-14