外观
论文arXiv 2609.125491 个月前
STAR：灵巧操作的视觉-触觉-语言-动作模型中的稀疏触觉表示学习
原文链接：STAR: Sparse Tactile Representation Learning in Vision-Tactile-Language-Action Models for Dexterous Manipulation（arXiv 2609.12549 · 发布日期：2026-09-11）
STAR: Sparse Tactile Representation Learning in Vision-Tactile-Language-Action Models for Dexterous Manipulation
为解决灵巧操作中稀疏触觉表示难题，STAR用联合预训练和稀疏触觉预测，在自建数据与真实任务上验证。
核心要点
- 灵巧操作需多指协同与有效触觉反馈，但真实大规模数据不足且触觉信号稀疏。
- 构建机器人平台和遥操作系统，采集200小时双手灵巧操作数据，含视觉、触觉和语言标注。
- STAR通过视觉-触觉联合预训练、稀疏全局触觉token和未来触觉预测处理稀疏性。
- 在四个真实世界任务中，经每任务100条轨迹后训练，平均成功率达61%。
验证范围与局限：仅报告在四个真实世界任务、每任务100条后训练轨迹下的任务特定后训练结果。
英文摘要 (Abstract)
Dexterous manipulation requires coordinated multi-finger control and effective tactile feedback, yet learning these capabilities remains challenging due to the lack of large-scale real-world data and the difficulty of extracting effective representations from sparse tactile signals. We build a robot platform and teleoperation system to collect a 200-hour bimanual dexterous manipulation dataset with synchronized visual, tactile, and language annotations, comprising 10,576 trajectories across 65 tasks, 69.5% of which involve dexterous multi-finger manipulation. We further propose STAR, an integrated training recipe for vision-tactile-language-action (VTLA) models that addresses the spatial, temporal, and informational sparsity of tactile signals through visual-tactile joint pre-training, sparse-global tactile token representation, and sparse future tactile prediction. Trained on this dataset, STAR achieves a 61% average success rate across four real-world tasks with 100 post-training trajectories per task, demonstrating dexterous performance under task-specific post-training.
作者：Xiangcheng Liu, Tianhao Wu, Le Zheng, Yidong Wang, Bowen Jiang, Mingjie Pan, Xinlin Ren, Yi Liu, Jianlan Luo
来源
本页事实对应的公开页面，以原文为准。