论文HF Daily Papers · arXiv 2610.392785 天前

WorldFlow联合预测视频与3D轨迹，空间推理误差降低54%

提出WorldFlow生成式世界模型，通过统一MoT架构联合预测未来视频与3D轨迹。在144万样本上预训练，其SpatialCog空间推理ADE降至0.060m，较最强基线减少54.2%，有效解锁视频模型物理常识。

AI 整理

仿真与训练
具身模型

本文由 AI 根据公开内容整理，可能有误，请以原文为准：WorldFlow: Joint Video and 3D Motion Prediction via Large-Scale Pretraining（HF Daily Papers · arXiv 2610.39278）

核心内容

针对现有视频世界模型3D动力学隐式化及动作预测绑定特定构型的问题，研究团队发布WorldFlow生成式世界模型。该模型将3D运动参数化为与视频网格对齐的连续位移token，采用混合Transformer（MoT）骨干网在整流流目标下联合去噪视频与3D轨迹模态。通过在包含仿真与真实世界的144万物体级样本异构数据集上预训练，实现了显式几何动力学与视觉演化的耦合。

为验证架构有效性，团队构建了含1678个样本、覆盖11个数据源的WorldFlowBench评测集。实验显示，WorldFlow在所有测试集中取得最低平均位移误差（ADE）和最终位移误差（FDE）。特别是在高难度SpatialCog空间推理任务中，ADE达0.060m，较最强外部基线降低54.2%；相比纯光流变体，ADE和FDE分别下降55.9%和68.5%，证实了联合建模对物理常识学习的显著增益。

关键看点

统一架构：采用Mixture-of-Transformers骨干网，将3D运动参数化为连续位移token，在整流流目标下联合去噪视频与轨迹模态
预训练规模：使用涵盖仿真与真实世界的144万物体级异构样本进行大规模预训练，实现跨域物理动力学学习
空间推理性能：在SpatialCog测试集ADE降至0.060m，较最强外部基线减少54.2%，突破纯视觉模型物理理解瓶颈
联合建模增益：相比flow-only变体，联合预测使ADE和FDE分别降低55.9%和68.5%，验证显式几何约束的有效性

工程与产业观察

WorldFlow通过将3D轨迹显式嵌入视频生成过程，解决了具身智能中视觉预测缺乏物理一致性的痛点。其MoT架构与百万级异构预训练范式，为构建具备空间推理能力的通用世界模型提供了可复现路径，对机器人长程任务规划与仿真数据增强具有重要工程参考价值。

英文摘要 (Abstract)

Predicting future physical states stands as a central challenge in embodied intelligence. Existing video world models leave 3D dynamics implicit, action-based formulations remain tied to specific robot configurations, and 3D motion prediction models use flow-only supervision, providing a limited learning signal for physical dynamics. To circumvent these limitations, we introduce , a generative world model that jointly predicts future video and query-based 3D trajectories. By parameterizing 3D motion as continuous displacement tokens aligned with the video grid, a unified Mixture-of-Transformers (MoT) backbone jointly denoises both modalities under a rectified-flow objective. This formulation couples explicit geometric dynamics with visual evolution, effectively unlocking the physical commonsense embedded within pretrained video models. To empirically scale and validate this unified architecture, we train on a heterogeneous mixture of 1.44 million object-level samples spanning simulated and real-world domains. We further introduce WorldFlowBench, comprising 1,678 evaluation samples from 11 data sources across simulated and real-world interactions, including spatial reasoning tasks. Across all three evaluation sets, WorldFlow achieves the lowest average displacement error (ADE) and final displacement error (FDE) among the evaluated learning-based methods. Notably, on the challenging SpatialCog set, WorldFlow achieves an ADE of 0.060 m, a 54.2% reduction compared with the strongest external baseline. It also reduces ADE and FDE by 55.9% and 68.5%, respectively, compared with its flow-only variant. Project page: https://worldflow-rbs.github.io/.

Hugging Face Daily Papers 热度：6 upvotes 作者：Yuhao Lu, Jiongxu Chen, Hexiang Wei, Jiajin Wu, Juelin Li, Mingjian Liang, Fan Yang, Zhuowei Liang, Zihao Liu, Zhennan Wang, Sheng Wang, Lin Shao