# YANAMI发布40万轮多模态状态交互数据集与基准

> 原文链接：https://robotics.hellobike.com/papers/2026-11-03-2611-05280-5062c8d1
> 频道：papers | 更新日期：2026-11-03 | 标签：auto, arxiv, hf-daily-papers
> 核心摘要：针对具身交互中多轮状态追踪难题，发布含40万对话、260万轮次的YANAMI数据集及评测基准。微调BAGEL模型使状态交互能力提升13.23%，填补了统一多模态模型在动态状态维护与视觉实现方面的评估空白。

# YANAMI发布40万轮多模态状态交互数据集与基准

::: v-pre
> 本文由 AI 根据公开内容整理，可能有误，请以原文为准：[YANAMI: A Large-Scale Multimodal Dataset and Benchmark for Multiturn Stateful Interaction](https://arxiv.org/abs/2611.05280)（HF Daily Papers · arXiv 2611.05280）

## 核心内容

针对现有统一多模态模型缺乏多轮状态交互监督的问题，研究团队推出YANAMI数据集与基准。YANAMI-400K包含40万条多模态对话、超200万张图像及260万轮次对话，覆盖多图融合、图像编辑、世界建模与视觉推理四大场景，强制要求模型在多轮交互中持续追踪并更新对话状态。配套YANAMI-Bench含400条测试对话，联合评估状态理解与视觉实现能力。

实验验证显示，在YANAMI-400K上微调BAGEL模型，其图像编辑、生成及多模态状态交互性能分别提升10.3%、2.0%和13.23%，且保持通用视觉理解能力。基准测试揭示当前主流模型在维护、更新及忠实呈现演化中的多模态状态方面仍存在显著短板。该工作为具身智能体在长程任务中的状态记忆与指令执行提供了关键数据支撑与量化评测标准。

## 关键看点

- **数据规模**：YANAMI-400K含40万对话、260万轮次及200万+图像，专攻多轮状态追踪
- **评测维度**：YANAMI-Bench联合评估对话状态理解与视觉实现，量化模型状态维护差距
- **训练增益**：微调BAGEL模型使多模态状态交互能力提升13.23%，图像编辑提升10.3%
- **场景覆盖**：涵盖多图融合、图像编辑、世界建模与视觉推理四类需状态演化的交互任务

## 工程与产业观察

YANAMI解决了具身智能长程交互中状态丢失的核心痛点，为训练具备记忆与状态更新能力的多模态大模型提供高质量SFT数据。其揭示的“理解-实现”鸿沟表明，仅靠单轮数据无法支撑复杂机器人任务规划，行业需转向状态感知的多轮对齐范式。

## 英文摘要 (Abstract)

Unified Multimodal Models (UMMs) integrate visual understanding, generation, and editing within a single unified framework. However, existing unified multimodal datasets and benchmarks remain dominated by single-turn tasks. Even in recent multi-turn datasets, many turns can be solved independently without tracking information across turns. Consequently, they provide limited supervision and evaluation for multimodal stateful interaction, which requires models to understand, maintain, update, and faithfully realize evolving dialogue states across turns. To address this gap, we introduce YANAMI, the first large-scale multimodal dialogue dataset and benchmark for stateful interaction, comprising YANAMI-400K and YANAMI-Bench. YANAMI-400K contains 400K multi-turn multimodal dialogues, with more than 2M images and 2.6M dialogue turns. The dialogues span four representative scenarios: Multi-Image Fusion, Image Editing, World Modeling, and Visual Reasoning, and require models to continuously track, preserve, and revise evolving multimodal dialogue states across turns. YANAMI-Bench comprises 400 multi-turn dialogues for systematically evaluating multimodal stateful interaction. It jointly evaluates Dialogue State Understanding and Visual Realization, characterizing the gap between models' ability to understand evolving dialogue states and their ability to faithfully realize them in visual outputs. Experiments show that fine-tuning BAGEL on YANAMI-400K improves image editing, generation, and multimodal stateful interaction by 10.3%, 2.0%, and 13.23%, respectively, while preserving its general visual understanding capabilities. Evaluations on YANAMI-Bench further reveal that existing UMMs still struggle to maintain, update, and visually realize evolving multimodal dialogue states.

Hugging Face Daily Papers 热度：26 upvotes
作者：Shengyu Wang, Tengyue Nong, Junming Liu, PENGHAOSONG, Siyu Li, Ruiqi Lei, Fushuo Huo, Yuhao Pan, Yunfeng Fan, Ding Wang, Nan Cheng, Haozhao Wang, Wenchao Xu
:::

## 原始参考来源 (Sources)

- [HF Daily Papers · arXiv 2611.05280：YANAMI: A Large-Scale Multimodal Dataset and Benchmark for Multiturn Stateful Interaction](https://arxiv.org/abs/2611.05280)