# PhysTacGen：物理感知的视觉-触觉传感器图像生成

> 原文链接：https://robotics.hellobike.com/papers/2026-10-06-2610-08068
> 频道：papers | 更新日期：2026-10-06 | 标签：arxiv, cs.RO
> 核心摘要：PhysTacGen结合材料描述与几何条件生成光学触觉图像，并在整理的SSVTP数据和下游代理任务上验证。

# PhysTacGen：物理感知的视觉-触觉传感器图像生成

::: v-pre
> 原文链接：[PhysTacGen: Physics-Aware Visual-Tactile Sensor Image Generation](https://arxiv.org/abs/2610.08068)（arXiv 2610.08068 · 发布日期：2026-10-06）

**PhysTacGen: Physics-Aware Visual-Tactile Sensor Image Generation**

PhysTacGen结合材料描述与几何条件生成光学触觉图像，并在整理的SSVTP数据和下游代理任务上验证。

## 核心要点

- 成对视觉—触觉数据采集成本高，视觉到触觉合成受材料属性差距和空间错位影响。
- 提出GTPO优化视觉语言模型生成结构化材料描述，并结合配对筛选与相对深度估计。
- SDXL ControlNet以RGB、相对深度和GTPO文本为条件，合成光学触觉图像。
- 实验显示SSVTP上结构相似性优于基线，盲测更偏好GTPO描述，且代码将开源。

**验证范围与局限**：验证范围包括整理的SSVTP数据、盲测用户研究和属性派生的力系数预测代理任务。

## 英文摘要 (Abstract)

Realistic physical interaction is a cornerstone of embodied intelligence, yet collecting paired visual--tactile data remains costly. Visual-to-tactile synthesis offers a promising approach to augmenting such data, but learning this mapping is complicated by the gap between visual appearance and contact-related material properties, as well as spatial misalignment in paired observations. To address these challenges, we present \textbf{PhysTacGen}, a visual-to-optical-tactile image generation framework that integrates material-aware descriptions with geometric conditioning. First, we introduce Group Tactile Policy Optimization (GTPO), a reinforcement learning strategy that refines a vision--language model to generate structured material descriptions using task-specific rewards. Second, we combine DINOv2-based pair curation with monocular relative-depth estimation to select training pairs and provide geometric priors. Finally, an SDXL ControlNet synthesizes optical tactile images conditioned on RGB, relative depth, and GTPO-generated text. Experiments on curated SSVTP data demonstrate improved structural similarity over the compared baselines, while a blinded user study shows a preference for GTPO-generated descriptions. Generated tactile inputs also improve performance on an attribute-derived force-coefficient prediction proxy. Together, these results demonstrate the effectiveness of PhysTacGen for optical tactile image synthesis and its utility in the evaluated downstream task.The code will be available at https://github.com/VDIGPKU/PhysTacGen.

作者：Guo Tang, Yongtao Wang
:::

## 原始参考来源 (Sources)

- [arXiv 2610.08068：PhysTacGen: Physics-Aware Visual-Tactile Sensor Image Generation](https://arxiv.org/abs/2610.08068)