外观
论文HF Daily Papers · arXiv 2610.337184 天前
RoboJev-27B开源：机器人操作成功检测达95.2%准确率
本文由 AI 根据公开内容整理，可能有误，请以原文为准：RoboJev: An Open Vision-Language Judge for Robot Success Detection（HF Daily Papers · arXiv 2610.33718）
核心内容
针对机器人操作视频缺乏任务完成标注的痛点，研究团队发布开源权重模型RoboJev-27B。该模型基于JEV-27B-VL变体，利用52,598条episode记录进行训练，专门用于根据任务指令及视频帧预测操作成败。在同型号SO-101机器人的268个保留测试集中，其平衡准确率达到95.2%，AUROC高达0.993，显著优于仅使用4帧输入的API类前沿VLM方案，为自动化数据清洗与评估提供了高精度工具。
技术架构上，RoboJev-27B采用Rank-32 LoRA适配器仅微调语言模型而冻结视觉编码器，并引入源特定KL惩罚以锚定研究数据分布。推理端采用校准双token读出机制，单次前向传播即可输出成功/失败概率。在未见过的实验室或机器人数据上AUROC仍达0.890，社区标注数据准确率为93.3%。工程效能方面，单张H100显卡中位推理耗时仅0.26秒，每千次预测成本约0.19美元，且完整开源了包含20,628条预测记录的透明评测基准。
关键看点
- 核心性能：SO-101测试集平衡准确率95.2%，AUROC 0.993；跨域泛化AUROC 0.890，社区数据准确率93.3%
- 训练策略：基于JEV-27B-VL使用Rank-32 LoRA微调LLM，冻结视觉编码器，引入源特定KL惩罚优化数据分布对齐
- 推理效能：单H100中位耗时0.26秒/episode，双token读出机制单次前向传播输出概率，千次预测成本约$0.19
- 开源资产：开放模型权重、推理代码及含20,628条预测、29个模型协议条目的透明评测集，支持可复现基准测试
工程与产业观察
RoboJev-27B解决了具身智能海量视频数据自动标注与质量筛选的工程瓶颈。其低成本、高精度及跨域泛化能力，使其可直接集成于数据采集管线或RLHF反馈回路中。相比通用大模型，专用Judge模型在垂直任务上的确定性优势，为构建大规模高质量机器人操作数据集提供了可落地的基础设施参考。
英文摘要 (Abstract)
Many robot manipulation videos do not indicate whether the task was successfully completed. RoboJev-27B is an open-weight vision-language model that takes a task instruction, 10 frames of a manipulation attempt, and (if available) final frames from auxiliary cameras and predicts whether the robot accomplished the task. RoboJev-27B is a variant of JEV-27B-VL, trained with a rank-32 LoRA adapter on the language model only (keeping the vision encoder frozen) using 52,598 episode records. It also uses a source-specific KL penalty that applies only to research data, anchoring predictions closer to the original model on those sources. It uses a calibrated two-token readout that outputs success/failure probabilities in a single forward pass. On 268 held-out episodes of the same robot model (SO-101), RoboJev-27B attains 95.2% balanced accuracy and 0.993 AUROC. It also generalises to episodes collected by unseen labs/robots (AUROC = 0.890 over 459 episodes) as well as to clearly labelled community-collected episodes (accuracy = 93.3% over 135 episodes), where the base model attains an AUROC = 0.900 on the unseen-lab/robot subset with the same inputs. It takes 0.26 seconds per episode (median, serial) to infer on a single H100, which is about \0.19 per 1,000 success predictions at 2.69 per GPU-hour (not including media processing and serving costs). RoboJev-27B is released together with its inference code, along with a transparent evaluation: 20,628 predictions organized into 29 model/protocol entries are made available to facilitate browsing and to enable reproducible benchmarking of success detection. We also compare against strong API-based frontier VLMs, which take in fewer input frames (4 images from the primary camera only); note that this comparison is not strictly like-for-like because RoboJev-27B is given more frames at inference.
Hugging Face Daily Papers 热度：1 upvotes 作者：Josef Liyanjun Chen
来源
本页事实对应的公开页面，以原文为准。