外观
论文arXiv 2608.024282 个月前
DF$^3$: 通过无解码器特征预测进行自主导航中的世界建模
原文链接：DF$^3$: World Modeling via Decoder-Free Feature Forecasting in Autonomous Navigation（arXiv 2608.02428 · 发布日期：2026-08-03）
DF$^3$: World Modeling via Decoder-Free Feature Forecasting in Autonomous Navigation
DF$^3$针对视频未来状态预测开销问题，在潜空间无解码器预测特征，并在公开基准和机器人仿真器零样本部署中验证。
核心要点
- 像素级生成方法易关注任务无关细节，潜特征方法仍受重型解码器瓶颈限制。
- DF$^3$在冻结视觉基础模型末端注入可学习空间查询，直接提取未来状态表示。
- MACF结合粗粒度光流扭曲与细粒度潜空间互相关，对齐并预测下一帧特征。
- 任务查询直接探测预测特征，在公开基准和仿真零样本部署中表现接近SOTA且更高效灵活。
验证范围与局限：验证范围为公开基准实验和机器人仿真器中的零样本部署。
英文摘要 (Abstract)
Forecasting future states from video sequences is a critical challenge for autonomous robotic systems and a fundamental objective of world modeling. Prior generative methods operating at the pixel level inevitably overemphasize task-irrelevant details, leading to prohibitive computational overhead. While latent-based approaches attempt to mitigate this by predicting features directly, the persistent reliance on heavy decoders for state-to-task mapping remains a computational bottleneck. In this work, we propose Decoder-Free Feature Forecasting (DF$^3$), a novel framework that models world evolution entirely within the latent space and directly derives task outputs, completely eliminating the need for a decoder. Specifically, DF$^3$ injects learnable spatial queries into the terminal blocks of a frozen vision foundation model to extract future state representations directly. By employing a lightweight, unified Motion-Aware Context Fusion (MACF) mechanism that seamlessly integrates coarse flow warping with fine-grained latent cross-correlation, these queries interact with historical token representations to explicitly align and forecast the feature of the next frame. Subsequently, a specialized set of task queries probes these forecasted features for the downstream task. Extensive experiments on public benchmarks and zero-shot deployment in a robotic simulator demonstrate that DF$^3$ achieves performance comparable to state-of-the-art methods while offering superior efficiency and flexibility for integrated perception and control.
作者：Jiaming Chen, Guoan Xu, Aoshen Huang, Haozhuo Zhang, Yang Li, Wei Pan
来源
本页事实对应的公开页面，以原文为准。