外观
论文arXiv 2609.196131 个月前
TacSushi：用于灵巧寿司操作的触觉扎根世界-动作建模
原文链接：TacSushi: Tactile-Grounded World-Action Modeling for Dexterous Sushi Manipulation（arXiv 2609.19613 · 发布日期：2026-09-17）
TacSushi: Tactile-Grounded World-Action Modeling for Dexterous Sushi Manipulation
TacSushi用触觉融合与训练期未来后果预测学习寿司灵巧操作，并在真实机器人试验中验证。
核心要点
- 面向食物变形、遮挡和接触不确定下的灵巧操作控制难题。
- 以Cosmos3为基础，编码RGB、语言和手状态，并门控融合指尖触觉特征。
- 训练期解码器预测未来视觉、进度、接触风险和触觉摘要，部署时移除。
- 完整TacSushi达68.3%分布内、37.5%分布外成功率，优于两个消融设置。
验证范围与局限：验证限于真实机器人寿司操作：三项分布内任务和两种分布外食材变体的600次rollout。
英文摘要 (Abstract)
Dexterous food manipulation requires control under deformation, occlusion, and uncertain contact. We present TacSushi, a tactile-grounded, Cosmos3-based world-action policy that learns from recorded future consequences while acting on current observations. The backbone encodes current RGB, language, and hand state, and feature-wise gated fusion incorporates fingertip tactile features into the action representation. During training, a decoder conditioned on demonstrated action chunks predicts logged future visual observations, task progress, relative contact risk, and tactile summaries; this decoder is removed at deployment. Failed trials provide consequence supervision, but their actions are excluded from imitation. We train TacSushi on 340 successful and 50 failed real-robot trials and compare six methods in 600 separate rollouts across three in-distribution tasks and two out-of-distribution ingredient variants. To assess food quality beyond a single geometric threshold, we score terminal outcomes using an anchored visual-quality protocol that equally weights five human ratings and three vision-language-model ratings per rollout. Full TacSushi achieves 68.3% average in-distribution success and 37.5% out-of-distribution success, compared with 36.7%/10.0% without future-consequence supervision and 25.0%/17.5% with direct tactile concatenation in place of gated fusion. These comparisons support complementary benefits of feature-wise gated tactile fusion and training-only predictive supervision.
作者：Haodi Hu, Kaen Kogashi, Toshiaki Koike–Akino
来源
本页事实对应的公开页面，以原文为准。