外观
论文arXiv 2610.0174227 天前
World Motion Models：SE(3)轨迹的灵活序列建模
原文链接：World Motion Models: Flexible Sequence Modeling of SE(3) Trajectories（arXiv 2610.01742 · 发布日期：2026-10-01）
World Motion Models: Flexible Sequence Modeling of SE(3) Trajectories
提出WMM，以稀疏SE(3)位姿轨迹和流匹配建模动态3D世界，并在6类3D视觉与机器人应用中验证。
核心要点
- 空间智能需要能描述动态3D世界时空变化的综合生成先验。
- WMM用稀疏刚体SE(3)位姿轨迹统一表示多类动态实体。
- 通过流匹配、逐token噪声和上下文token支持任意条件化。
- 在6类3D视觉与机器人应用中展示了通用性、灵活性和强性能。
验证范围与局限：验证范围为6类3D视觉与机器人应用；摘要未说明具体仿真或真机设置。
英文摘要 (Abstract)
Equipping artificial agents with spatial intelligence requires a comprehensive generative prior over the dynamic 3D world. We propose World Motion Models (WMMs) that capture "what was, is, and will be where across time" via sparse SE(3) pose trajectories. WMMs are built on the observation that elements of dynamic scenes can be well approximated by a set of rigid SE(3) trajectories, a minimal yet expressive primitive for 4D modeling. This representation unifies articulated objects, human bodies, hand-object interactions, piecewise-rigid scene dynamics, camera motion, and even robot states and actions into a single shared space. Given this representation, we cast the joint distribution of these entities as a flexible sequence modeling problem, utilizing flow-matching with per-token noise levels. Coupled with a context token mechanism for non-sequential conditioning, this formulation supports any-to-any marginal conditioning across an arbitrary number of entities and time steps. Tasks such as future prediction, motion infilling, model-predictive control, inverse kinematics, cross-embodiment retargeting, and policy learning all reduce to the application of different masks over the same network. Experiments on 6 diverse applications of 3D vision and robotics demonstrate the versatility and flexibility of WMMs with strong performance.
作者：Jinyu Lei, Qianqian Wang, Trevor Jackson Darrell, Angjoo Kanazawa
来源
本页事实对应的公开页面，以原文为准。