# RoboJudge-9B发布：具身视频生成评测基准与对齐人类判断的专用模型

> 原文链接：https://robotics.hellobike.com/papers/2026-10-20-2610-27782-02472909
> 频道：papers | 更新日期：2026-10-20 | 标签：auto, arxiv, hf-daily-papers
> 核心摘要：研究推出RoboJudge-Bench评测基准与12K视频训练集，微调9B MLLM实现具身视频物理与指令对齐评估。该模型与人类评分Pearson相关性达0.719，显著优于Gemini-3.7-Flash等通用大模型，填补了具身世界模型自动化评测缺口。

# RoboJudge-9B发布：具身视频生成评测基准与对齐人类判断的专用模型

::: v-pre
> 本文由 AI 根据公开内容整理，可能有误，请以原文为准：[RoboJudge: Multimodal Language Models as Judges for Embodied Video Generation](https://arxiv.org/abs/2610.27782)（HF Daily Papers · arXiv 2610.27782）

## 核心内容

针对多模态大模型在具身视频生成评测中与人类判断不一致的问题，研究团队发布RoboJudge-Bench基准与RoboJudge-9B专用评估模型。基准涵盖800条人工标注视频，覆盖夹爪、人手及灵巧手等10种生成器；基于12K视频与25K验证分数构建RoboJudge-Train数据集，通过监督学习与强化学习微调9B参数模型，使其在物理遵循与指令对齐维度上达到最高人机一致性。

实验显示通用MLLM在具身场景物理推理能力不足，而RoboJudge-9B在RoboJudge-Bench上与人类评分Pearson相关系数达0.719，较最优基线Gemini-3.7-Flash提升0.089，且在VideoPhy2等外部基准泛化良好。该模型可作为下游自动化评估器，量化衡量视频生成模型在物理有效性与任务忠实度上的表现，为具身世界模型迭代提供可靠反馈信号。

## 关键看点

- **评测基准**：RoboJudge-Bench包含800条人工标注视频，覆盖夹爪、人手及灵巧手三类末端执行器与10种视频生成器，聚焦物理遵循与指令对齐双维度评估。
- **训练数据**：构建RoboJudge-Train数据集含12K视频与25K人工验证分数及模型辅助推理依据，支持监督学习与强化学习两阶段微调以对齐人类偏好。
- **性能指标**：RoboJudge-9B与人类评分Pearson相关性达0.719，较Gemini-3.7-Flash提升0.089，在VideoPhy2和Physion-Eval等外部物理推理基准亦取得增益。
- **应用价值**：作为自动化评估器可量化视频生成模型的物理有效性与任务忠实度，解决通用MLLM在具身场景下物理推理迁移失效问题，支撑世界模型闭环迭代。

## 工程与产业观察

该工作确立了具身视频生成自动化评测的数据标准与模型范式，证明专用微调MLLM可替代昂贵人工评估。其构建的物理与指令对齐评分体系，为具身智能从仿真到现实的Sim-to-Real验证提供了可量化的中间层指标，对加速世界模型训练收敛与降低数据标注成本具有直接工程参考价值。

## 英文摘要 (Abstract)

Video generation models are promising foundations for embodied world models, but only if their generated rollouts are physically valid and faithful to task instructions. Multimodal large language models (MLLMs) are increasingly used to judge these properties, yet their agreement with human judgments on embodied videos has not been systematically evaluated. To address this gap, we introduce RoboJudge-Bench, a benchmark for evaluating MLLMs as judges of embodied video generation. The benchmark measures two fundamental dimensions, Physical Adherence and Instruction Alignment, across 800 human-annotated videos from 8 source corpora and 10 video generators, spanning grippers, human hands, and dexterous hands. Experiments reveal uneven alignment between existing judges and human ratings: general-purpose MLLMs show lower agreement on physical adherence, while specialized video judges designed for general-domain physical reasoning do not consistently transfer to embodied scenarios. Motivated by these findings, we construct RoboJudge-Train, comprising 12K videos annotated with 25K human-verified scores and model-assisted rationales for Physical Adherence and Instruction Alignment. We fine-tune a 9B MLLM on this dataset with supervised learning followed by reinforcement learning, yielding RoboJudge-9B. Among the model judges evaluated on RoboJudge-Bench, RoboJudge-9B achieves the highest overall agreement, reaching a Pearson correlation of 0.719 with a statistically significant 0.089 gain over Gemini-3.7-Flash, the best-performing proprietary baseline in our evaluation. Training on RoboJudge-Train also yields gains on external benchmarks, including VideoPhy2 and Physion-Eval. We further use RoboJudge-9B as a downstream evaluator to benchmark video generation models according to Physical Adherence and Instruction Alignment.

Hugging Face Daily Papers 热度：2 upvotes
作者：Siyuan Ma, Yu Qi, Cheng Ye, Ruotian Peng, Xinyi Xu, Zitiantao Lin, Zedong Cai, Yijian Huang, Weiyang Liu, Siyuan Wang, Yandong Wen
:::

## 原始参考来源 (Sources)

- [HF Daily Papers · arXiv 2610.27782：RoboJudge: Multimodal Language Models as Judges for Embodied Video Generation](https://arxiv.org/abs/2610.27782)