外观
论文arXiv 2609.209801 个月前
ForeTac-VLA：面向接触密集型机器人操作的基于预测的触觉-视觉-语言-动作模型
原文链接：ForeTac-VLA: A Forecasting-Based Tactile-Vision-Language-Action Model for Contact-Rich Robotic Manipulation（arXiv 2609.20980 · 发布日期：2026-09-17）
ForeTac-VLA: A Forecasting-Based Tactile-Vision-Language-Action Model for Contact-Rich Robotic Manipulation
ForeTac-VLA通过预测未来触觉状态融合触觉、视觉与语言，在真实接触密集操作任务上提升动作生成鲁棒性。
核心要点
- VLA依赖视觉感知，在关键物理交互状态不可见的接触密集环境中鲁棒性受限。
- 模型将近期触觉编码为时序表示，经双向交叉注意力与视觉语言特征融合。
- Transformer预测多步未来触觉状态，使模型联合利用已观测和预期接触来生成动作。
- 四个真实接触密集操作任务中平均成功率达95%，并在低照度和视觉杂乱条件下保持强表现。
验证范围与局限：验证范围为四个真实世界接触密集型机器人操作任务。
英文摘要 (Abstract)
Vision-language-action (VLA) models have demonstrated strong capabilities in robotic manipulation, yet their reliance on visual perception limits robustness in contact-rich environments, where critical physical interaction states may not be visually observable. Existing tactile-enhanced VLA methods improve physical grounding using observed tactile feedback, but most remain largely reactive rather than explicitly modeling how contact may evolve. Therefore, we propose ForeTac-VLA, a forecasting-based tactile-vision-language fusion model that predicts future tactile states to guide action generation. Specifically, ForeTac-VLA encodes recent tactile observations into temporal representations and integrates them with vision-language features through bidirectional cross-attention. Further, a transformer-based forecasting module predicts multi-step future tactile states, enabling the model to reason jointly over observed and anticipated contact. Finally, the fused multimodal representations and predicted future tactile states are fed into the VLA backbone to condition action generation. To stabilize training, a ground-truth-to-prediction curriculum is employed when early forecasts are unreliable. Across four real-world contact-rich manipulation tasks, ForeTac-VLA achieves an average success rate of 95%, outperforming the fine-tuned VLA model by 36.25 percentage points and state-of-the-art tactile-enhanced VLA baselines by over 22 percentage points. ForeTac-VLA also maintains strong performance under low-illumination and visually cluttered conditions. Video demonstrations can be found on https://foretac-vla.github.io/
作者：Zhengyu Tao, Xin Li, Xin Wang
来源
本页事实对应的公开页面，以原文为准。