外观
论文arXiv 2608.213552 个月前
ViTacPhys：从人类视觉-触觉演示中感知物理属性的抓取
原文链接：ViTacPhys: Physical Property-Aware Grasping from Human Visual-Tactile Demonstrations（arXiv 2608.21355 · 发布日期：2026-08-21）
ViTacPhys: Physical Property-Aware Grasping from Human Visual-Tactile Demonstrations
ViTacPhys从人类视觉-触觉演示估计物体物理属性并条件化抓取策略，在属性预测和真实自适应抓取中验证。
核心要点
- 现有视觉动作模型很少显式利用物体物理属性来调整操控策略。
- 该框架结合时序视觉-触觉建模、跨注意力多模态融合和视觉语言模型语义先验。
- 在已见与留出物体上评估质量、摩擦和刚度估计，并报告较高分类精度和较低刚度误差。
- 通过有限机器人遥操作数据、机器人风格视频增强和匹配动作演示迁移到在线自适应抓取。
验证范围与局限：验证范围包括60个刚性和可变形物体的属性估计，以及分布内和分布外物体的真实自适应抓取。
英文摘要 (Abstract)
Recent vision-based action models have demonstrated strong capabilities in complex manipulation, but they rarely leverage explicit object physical properties to adapt their policies. We introduce ViTacPhys, a visual-tactile framework and data acquisition system that estimates object mass and friction-coefficient classes, together with continuous stiffness, from human manipulation demonstrations. Trained on data from 60 rigid and deformable objects, ViTacPhys combines temporal visual-tactile modeling, cross-attention multimodal fusion, and a semantic prior derived from a vision-language model. On seen objects, it achieves 97.2% mass classification accuracy, 98.8% friction-coefficient classification accuracy, and a stiffness mean absolute percentage error (MAPE) of 5.51%. On held-out objects from known categories, it achieves 87.5% mass accuracy, 97.5% friction-coefficient accuracy, and a stiffness MAPE of 9.08%. We transfer ViTacPhys from the human domain to the robot domain using limited robot teleoperation data, robot-style video augmentation, and human demonstrations with matched actions, and deploy it as an online module for adaptive grasping. The resulting physical-property-conditioned policy achieves total grasping success rates of 95.0% on in-distribution objects and 83.4% on out-of-distribution objects. For out-of-distribution objects successfully grasped by both methods, its force profiles are more consistent with human teleoperation than those produced by ACT. These results demonstrate the feasibility of explicitly estimating and conditioning on object physical properties for real-world adaptive grasping.
作者：Yiwen Liu, Yujun Zhu, Kui Jia, Zhao Liao, Yangwei You, Shuaijun Wang
来源
本页事实对应的公开页面，以原文为准。