具身开发者社区

外观

论文arXiv 2610.0762122 天前

TacZero：使用带触觉反馈的通用视觉语言模型进行免训练销插入 ​

TacZero用预训练通用VLM结合视觉、机器人状态与触觉反馈，在真实圆柱销插入任务中免训练选择动作。

AI 整理
  • 本体 SDK 与控制
  • 数据与采集
  • 具身模型

原文链接：TacZero: Training-Free Peg Insertion Using a General-Purpose Vision-Language Model with Tactile Feedback（arXiv 2610.07621 · 发布日期：2026-10-06）

TacZero: Training-Free Peg Insertion Using a General-Purpose Vision-Language Model with Tactile Feedback

TacZero用预训练通用VLM结合视觉、机器人状态与触觉反馈，在真实圆柱销插入任务中免训练选择动作。

核心要点 ​

  • 目标是让机器人依据语言指令和感知观测执行新的富接触操作，无需任务特定训练或手写规则。
  • TacZero将相机图像、机器人状态和三轴触觉响应叠加给VLM，由其生成末端目标位置与夹爪开合命令。
  • 在真实世界圆柱销插入实验中，使用数值触觉输入成功15/20次，无触觉输入成功10/20次。
  • 研究为通用VLM用于富接触操作提供具体起点，并指出继续推进该方向仍面临挑战。

验证范围与局限：验证范围为真实世界圆柱销插入实验，报告了有无触觉输入各20次试验。

英文摘要 (Abstract) ​

Robots that autonomously determine their actions from language instructions and sensory observations could perform new contact-rich manipulation tasks without task-specific training or hand-designed rules. To perform these tasks, robots must infer how objects contact one another and move as a result, then select actions. For contact inference and action selection, prior approaches involve designing estimation models and tactile feedback control laws, or learning models for object-motion estimation, action-outcome prediction, and action selection from tactile data. Instead, we propose TacZero, which uses a pretrained general-purpose vision-language model (VLM) to interpret visual and tactile observations and select robot actions without additional tactile or manipulation training or task-specific rules for contact interpretation or action selection. TacZero provides the VLM with camera images, robot state, and three-axis tactile responses represented as numerical values or vectors overlaid on the images. From these observations and interaction history, the VLM generates commands specifying target end-effector positions and gripper opening or closing, which a low-level controller executes. In real-world cylindrical-peg insertion experiments, TacZero succeeded in 15 of 20 trials with numerical tactile input, compared with 10 of 20 without tactile input. This study provides a concrete starting point for further research on contact-rich manipulation using general-purpose VLMs and highlights challenges in pursuing this direction.

作者：Kazutoshi Tanaka

来源

本页事实对应的公开页面，以原文为准。

  1. arXiv 2610.07621：TacZero: Training-Free Peg Insertion Using a General-Purpose Vision-Language Model with Tactile Feedback访问 2026-10-21