外观
论文arXiv 2610.2006414 天前
FoveaVLA：用 Foveation 动作替代 VLA 中的腕部相机
FoveaVLA针对腕部相机视野受限与外部相机细节丢失，预测交互区域并裁剪原图，在四个真实任务套件上验证。
Franka FR3原文链接：FoveaVLA: Foveation Actions Replace the Wrist Camera in VLAs（arXiv 2610.20064 · 发布日期：2026-10-14）
FoveaVLA: Foveation Actions Replace the Wrist Camera in VLAs
FoveaVLA针对腕部相机视野受限与外部相机细节丢失，预测交互区域并裁剪原图，在四个真实任务套件上验证。
核心要点
- 腕部相机有近距离细节但视野受限，外部相机全景输入低分辨率会丢失操作细节。
- 模型在每个动作块旁预测下次策略查询要观察的交互区域，并输入原图裁剪与全景视图。
- 基于π_{0.5}构建Single和Dual配置，并用DROID与RoboInter交互标注进行后训练。
- 四个真实任务套件中，Single达70.8%成功率，Dual达77.1%，高于腕部相机基线52.1%。
验证范围与局限：验证范围为四个真实世界任务套件，以及使用一个或两个外部立体相机的Single/Dual配置。
英文摘要 (Abstract)
We present FoveaVLA, a Vision-Language-Action (VLA) model that replaces the wrist camera with learned foveation actions. Wrist cameras provide close-up detail but restrict the view to the end-effector's surroundings. External cameras see the whole scene, but the policy receives them at low resolution, so the fine detail needed for manipulation is lost. FoveaVLA recovers this detail by predicting, alongside each action chunk, the interaction region to observe at the next policy query. The policy then receives crops of these regions, taken from the original-resolution images, together with the full views. We build FoveaVLA on $π_{0.5}$ in two configurations, Single and Dual, using one or two external stereo cameras with full and foveal RGB and depth inputs, and post-train both on DROID with RoboInter interaction annotations. Across four real-world task suites, FoveaVLA-Single achieves 70.8% success with one external RGB-D view, and FoveaVLA-Dual achieves 77.1%, versus 52.1% for the wrist-equipped $π_{0.5}$. Both match the baseline on plain pick-and-place and improve success with distractors, cloth manipulation, and far-away objects.
作者：Bingao Chen, Lihan Zha, C. Karen Liu, Guy Tevet
来源
本页事实对应的公开页面，以原文为准。