外观
论文HF Daily Papers · arXiv 2610.316236 天前
F2G-Pose：单次前向RGB-D类别级位姿估计达28FPS
本文由 AI 根据公开内容整理，可能有误，请以原文为准：F2G-Pose: Geometry-Aware Foundation Feature Lifting for Direct RGB-D Category-Level Object Pose Estimation（HF Daily Papers · arXiv 2610.31623）
核心内容
针对类别级RGB-D物体位姿估计中中间结构误差传播及扩散模型推理慢的痛点，研究团队提出F2G-Pose实时直接估计框架。该方法无需实例CAD模型，仅凭单帧RGB-D输入即可在一次前向传播中输出6D位姿、度量尺寸及相机坐标系下的完整形状。在仅使用合成SOPE数据训练任务网络的情况下，于SOPE和ROPE基准取得SOTA精度，纯位姿推理速度达28 FPS，并在HouseCat6D及HANDAL真实数据集上验证了泛化能力。
技术架构上，F2G-Pose将冻结的RADIO视觉基础模型稠密特征提升至部分点云空间，利用DGCNN进行局部几何Token化编码，并通过混合专家Transformer整合物体级全局结构。联合尺寸预测与相机系形状补全模块引入稠密几何监督以增强度量一致性。相比GenPose++等候选生成方法，该端到端架构显著降低延迟并提升鲁棒性，项目主页已开源完整代码、预训练权重及实测操作视频，支持分段RGB-D输入下的具身抓取部署。
关键看点
- 实时推理性能：单次前向传播同时输出位姿、尺寸与形状，纯位姿推理达28 FPS，解决扩散模型高延迟瓶颈
- 几何特征融合：结合冻结RADIO基础模型与DGCNN局部Token化，经MoE Transformer整合全局结构提升表征能力
- 合成数据迁移：任务网络仅用合成SOPE数据训练，即在ROPE、HouseCat6D及HANDAL真实数据集验证强泛化性
- 开源资产完备：提供项目主页、源代码、预训练模型及真机操作实验数据，支持分段RGB-D输入的具身应用开发
工程与产业观察
F2G-Pose证明了“冻结视觉大模型+轻量几何解码器”范式在具身感知中的工程价值。其28 FPS实时性与端到端架构规避了传统NOCS或扩散模型的算力瓶颈，且仅依赖合成数据训练即可实现Sim-to-Real迁移。对于缺乏大规模真实标注数据的机器人抓取与操作任务，该方案提供了高精度、低延迟的类别级位姿估计基线，具备较高的端侧部署参考意义。
英文摘要 (Abstract)
This paper studies category-level RGB-D object pose estimation, which recovers an object's 3D rotation and translation without instance-specific CAD models or reference views. This task is highly ambiguous due to intra-class variations, symmetries, and partial observations. Current solutions struggle to balance reliability and efficiency: methods based on intermediate structures are prone to error propagation, while diffusion models suffer from high inference latency and complex post-hoc candidate extraction. To address this, we present F2G-Pose, a real-time direct, single-pass framework that predicts category-level pose, metric size, and camera-frame completed shape from a single RGB-D observation in one feed-forward pass. F2G-Pose lifts dense visual foundation features to partial point clouds, encodes local geometry with DGCNN tokenization, and integrates object-level structure with a Mixture-of-Experts Transformer. Joint size prediction and camera-frame shape completion encourage metric consistency through dense geometric supervision. With its task-specific network trained only on pose-annotated synthetic SOPE data and its frozen RADIO backbone externally pretrained, F2G-Pose achieves state-of-the-art accuracy on SOPE and ROPE while running at 28 FPS for pose-only inference, and further demonstrates strong camera-frame geometry on HouseCat6D and real-object transfer on HANDAL. Analyses of latency, robustness, shape completion, and GenPose++ candidate behavior further demonstrate the efficiency and reliability of F2G-Pose, with real-world manipulation experiments demonstrating deployment potential under segmented RGB-D inputs. The project page, code, and pretrained models are made available at https://jrryzh.github.io/F2G-Pose/.
Hugging Face Daily Papers 热度：13 upvotes 作者：Jinyu Zhang, Haitao Lin, Jiashu Hou, Xiangyang Xue, Yanwei Fu
来源
本页事实对应的公开页面，以原文为准。