外观
论文HF Daily Papers · arXiv 2610.316686 天前
视频基础模型综述：以世界建模统一读写交互与状态预测
本文由 AI 根据公开内容整理，可能有误，请以原文为准：Reading, Writing, Sharing, and Interacting with the World through Video（HF Daily Papers · arXiv 2610.31668）
核心内容
针对视频基础模型与世界模型目标差异，该综述提出以世界建模作为统一理论框架，将观测视频定义为潜在时序世界的部分、有损且时序有序的投影。文章围绕Reading（推断世界属性）、Writing（合成视频）、Sharing（整合理解生成机制）及Interacting（基于动作更新状态）四大互补操作重组文献体系，旨在解决当前视觉序列处理中观察与演化世界的割裂问题。
技术层面，该框架严格区分了观测/生成序列的时间连贯性与底层世界状态的一致性。综述系统追踪了表征、模型、数据集及基准测试的演进路径，确认研究范式已从传统的片段级识别与感知保真度，转向关注持久状态维持、可控演化、物理因果有效性以及基于交互和下游决策的评估体系，并总结了实现通用世界模型面临的开放性挑战。
关键看点
- 统一理论框架：将视频视为潜在时序世界的有损投影，通过读写共享交互四操作统合基础模型与世界模型研究
- 核心范式转移：研究重心从片段级识别与感知保真度，转向持久状态维持、可控演化及物理因果有效性验证
- 状态一致性解耦：明确区分观测或生成序列的时间连贯性与底层潜在世界状态的一致性，强化动态预测能力
- 评估体系重构：倡导建立基于交互反馈和下游具身决策任务的评测基准，替代纯视觉生成质量指标
工程与产业观察
该综述为具身智能提供了关键理论支撑，明确了视频模型从“看”到“懂物理”的演进路线。对于机器人算法工程师，文中关于持久状态与因果有效性的论述，直接指向了Sim-to-Real仿真器构建与端到端策略训练的核心痛点，是设计下一代具备物理常识的具身大模型的重要参考坐标。
英文摘要 (Abstract)
Video foundation models have advanced rapidly in understanding and generating visual sequences, while video world models seek to capture the underlying dynamics of the world they depict. Despite their different objectives, both concern the relationship between video observations and an evolving world. This survey adopts world modeling as a unifying lens for examining these developments. Viewing an observed video as a partial, lossy, and temporally ordered projection of a latent temporal world, we organize the literature around four complementary operations: reading infers world properties from observed video; writing synthesizes video from specified or learned world conditions; sharing integrates understanding and generation through common modeling mechanisms; and interacting predicts and updates world states under actions and evolving observations. This perspective distinguishes temporal coherence in observed or generated sequences from consistency in the underlying world state. Across tasks, representations, models, datasets, and benchmarks, we trace a shift in emphasis from clip-level recognition and perceptual fidelity toward persistent state, controllable evolution, physical and causal validity, and evaluation grounded in interaction and downstream decisions, and we distill the open challenges that follow.
Hugging Face Daily Papers 热度：47 upvotes 作者：Shengqiong Wu, LUO MENG, Bobo Li, Ziwei Liu, Shuicheng Yan, Ming-Hsuan Yang, Philip Torr, Hao Fei
来源
本页事实对应的公开页面，以原文为准。