Skip to content

论文HF Daily Papers · arXiv 2611.197717 天前

JEPA4R提出4D点云规划,零样本迁移真机操作成功率达98.4% ​

本文由 AI 根据公开内容整理,可能有误,请以原文为准:JEPA4R: Embodied JEPA World Models with 4D Point Planning(HF Daily Papers · arXiv 2611.19771)

核心内容 ​

针对现有JEPA世界模型依赖2D图像导致几何信息隐式耦合、且潜空间想象不可视的问题,研究提出JEPA4R架构。该模型直接观测单深度相机点云,在潜空间完成预测与4D点云规划,并通过冻结潜空间上的策略头输出动作。实验显示其解码器生成的4D点云在200步内平均点误差控制在1.1cm以内,有效解决了具身操作中几何度量缺失与模型可解释性难题。

在OGBench和CortexBench协议下,JEPA4R在Cube-Single/Double/Triple及Scene任务中领先像素级JEPA;在MetaWorld和Adroit Relocate任务中分别取得98.4%和88.0%成功率,显著优于R3M的96.0%和65.3%。该系统仅在数字孪生环境中训练,无需真实数据微调,即可零样本部署于真实机械臂,同时兼容平行夹爪与多指灵巧手末端执行器,验证了纯仿真训练到真机操作的有效迁移能力。

关键看点 ​

  • 4D点云规划:基于单深度相机点云在潜空间预测与规划,解码可视化4D点云200步平均误差仅1.1cm
  • 基准测试SOTA:MetaWorld与Adroit Relocate成功率达98.4%和88.0%,超越R3M基线2.4%与22.7%
  • 零样本真机迁移:仅依赖数字孪生训练,无需实机微调即可驱动夹爪与灵巧手完成真实物理操作
  • 潜空间解耦:通过冻结世界模型潜空间并挂载策略头,解决2D图像几何与外观纠缠导致的规划失效

工程与产业观察 ​

JEPA4R证明了以点云为原生表征的世界模型在保留精确几何度量方面的优势,克服了像素级JEPA在精细操作中的局限。其纯仿真训练至真机零样本迁移的成功,为降低具身智能数据采集成本提供了可行范式,对构建通用操作基础模型具有重要工程参考价值。

英文摘要 (Abstract) ​

Joint-embedding predictive architecture (JEPA) world models predict future embeddings instead of pixels and plan in latent space. Two challenges remain: 2D images leave the metric geometry that manipulation acts on implicit and entangled with appearance, and what these models imagine stays hidden, as they are mostly judged in simulation by short-horizon latent planning. We present JEPA4R, a JEPA world model that observes single-depth-camera point clouds, predicts and plans in their latent space, and acts through a policy head on its frozen latents; we call this 4D point planning. A decoder on the frozen world model shows what it imagines as 4D point clouds, with a mean point error within 1.1 cm over 200 steps. Under the policy-learning protocols of OGBench and CortexBench, JEPA4R leads pixel JEPA models on OGBench Cube-Single, Cube-Double, Cube-Triple and Scene, and ranks first on MetaWorld and Adroit Relocate with 98.4% and 88.0%, above the runner-up, R3M, at 96.0% and 65.3%. Trained only in a digital twin, JEPA4R and its policy heads run zero-shot on the real arm with both a gripper and a dexterous hand.

Hugging Face Daily Papers 热度:3 upvotes 作者:Jiarui Dong, zhiheng liang, Yanjun Li, Zeyu Zhang, Heejeong Nam, Haoyu Chen, Hao Tang

基于本页技术方向关联的整机参数、核心零部件、实测报告与同赛道选型横评。

外部来源

本页事实对应的公开页面,以原文为准。(点击将离开本站)

  1. HF Daily Papers · arXiv 2611.19771:JEPA4R: Embodied JEPA World Models with 4D Point Planning ↗访问 2026-11-18