论文/arXiv 2610.08068·22 天前

PhysTacGen：物理感知的视觉-触觉传感器图像生成

PhysTacGen结合材料描述与几何条件生成光学触觉图像，并在整理的SSVTP数据和下游代理任务上验证。

AI 整理

• 本体 SDK 与控制
• 感知与导航
• 数据与采集

原文链接：PhysTacGen: Physics-Aware Visual-Tactile Sensor Image Generation（arXiv 2610.08068 · 发布日期：2026-10-06）

PhysTacGen: Physics-Aware Visual-Tactile Sensor Image Generation

PhysTacGen结合材料描述与几何条件生成光学触觉图像，并在整理的SSVTP数据和下游代理任务上验证。

核心要点

• 成对视觉—触觉数据采集成本高，视觉到触觉合成受材料属性差距和空间错位影响。
• 提出GTPO优化视觉语言模型生成结构化材料描述，并结合配对筛选与相对深度估计。
• SDXL ControlNet以RGB、相对深度和GTPO文本为条件，合成光学触觉图像。
• 实验显示SSVTP上结构相似性优于基线，盲测更偏好GTPO描述，且代码将开源。

验证范围与局限：验证范围包括整理的SSVTP数据、盲测用户研究和属性派生的力系数预测代理任务。

英文摘要 (Abstract)

Realistic physical interaction is a cornerstone of embodied intelligence, yet collecting paired visual--tactile data remains costly. Visual-to-tactile synthesis offers a promising approach to augmenting such data, but learning this mapping is complicated by the gap between visual appearance and contact-related material properties, as well as spatial misalignment in paired observations. To address these challenges, we present \textbf{PhysTacGen}, a visual-to-optical-tactile image generation framework that integrates material-aware descriptions with geometric conditioning. First, we introduce Group Tactile Policy Optimization (GTPO), a reinforcement learning strategy that refines a vision--language model to generate structured material descriptions using task-specific rewards. Second, we combine DINOv2-based pair curation with monocular relative-depth estimation to select training pairs and provide geometric priors. Finally, an SDXL ControlNet synthesizes optical tactile images conditioned on RGB, relative depth, and GTPO-generated text. Experiments on curated SSVTP data demonstrate improved structural similarity over the compared baselines, while a blinded user study shows a preference for GTPO-generated descriptions. Generated tactile inputs also improve performance on an attribute-derived force-coefficient prediction proxy. Together, these results demonstrate the effectiveness of PhysTacGen for optical tactile image synthesis and its utility in the evaluated downstream task.The code will be available at https://github.com/VDIGPKU/PhysTacGen.

作者：Guo Tang, Yongtao Wang

来源

本页事实对应的公开页面，以原文为准。

1. arXiv 2610.08068：PhysTacGen: Physics-Aware Visual-Tactile Sensor Image Generation访问 2026-10-21