Skip to content

论文HF Daily Papers · arXiv 2611.229255 天前

Invoker-Render发布:4步因果流式渲染实现30分钟具身仿真零漂移 ​

本文由 AI 根据公开内容整理,可能有误,请以原文为准:Code World Model 2.0: Towards Infinite Controllable Video Generation(HF Daily Papers · arXiv 2611.22925)

核心内容 ​

针对具身智能视觉代理中显式世界状态模拟与视频合成解耦的挑战,研究团队发布Invoker-Render系统。该方案包含双向创建模型与因果四步流式模型,共享结构代理、外观参考及语言条件接口。基于约200小时多粒度对齐语料训练,系统可在运行时动态选择输入细节层级,既能在精细几何约束下保持保真度,也能在粗粒度代理下合成合理局部运动,为离线片段生成与实时交互流提供统一可控接口。

为解决流式推理中的步进压缩与自回归漂移问题,论文提出Progressive Forcing(ProForcing)训练策略。该方法先训练因果预测器并蒸馏为四步渲染器,随后切换至连续自生成轨迹进行微调,配合流水线服务栈实现高效推理。实验表明,该系统在不同代理族、粒度及相机路径下均展现稳定控制力,性能优于现有流式基线,并在30分钟赛车场景长程Rollout中保持场景与主体外观一致性,验证了其在长时序具身仿真中的工程可用性。

关键看点 ​

  • ProForcing策略:采用渐进强制训练法,先蒸馏因果预测器为4步渲染器,再基于自生成轨迹微调,消除长时序推理漂移
  • 多粒度代理兼容:共享条件接口支持从粗布局到精细关节几何的动态切换,运行时按需调整输入细节以平衡算力与精度
  • 长程稳定性验证:在30分钟赛车场景连续Rollout测试中保持场景与主体外观一致,突破流式视频生成的时间尺度限制
  • 数据与架构规格:基于200小时多粒度对齐语料训练,构建双向创建与因果流式双模型架构,适配实时交互低延迟需求

工程与产业观察 ​

该工作通过算法层面的蒸馏与渐进训练,解决了世界模型在具身仿真中长时序误差累积的核心痛点。4步因果渲染与多粒度代理机制,为机器人Sim-to-Real提供了兼顾实时性与物理一致性的视觉闭环验证工具,对降低真机试错成本具有直接工程价值。

英文摘要 (Abstract) ​

Visual proxies decouple explicit world-state simulation from photorealistic video synthesis, offering a controllable interface for both offline clip creation and real-time interactive streaming. Yet a practical renderer must address two challenges. Upstream proxies vary from coarse layouts to fine articulated geometry depending on the interaction, and streaming inference must operate in a few causal steps without accumulating drift across self-generated history. We present Invoker-Render, pairing a bidirectional creation model with a causal four-step streaming model under a shared conditioning interface over structural proxies, an appearance reference, and language. Trained on an aligned approximately 200-hour multi-granularity corpus, Invoker-Render honors fine geometric constraints when supplied while synthesizing plausible local motion under coarse proxies, enabling runtime selection of input detail. To bridge bidirectional creation and real-time streaming without compounding step compression and rollout drift, we introduce Progressive Forcing (ProForcing). ProForcing first trains a causal predictor and distills it into a four-step renderer using histories derived from training videos. Training then shifts to continuous self-generated rollouts. A pipelined serving stack supports efficient streaming inference. Experiments show versatile control across proxy families, granularities, and camera paths, consistent gains over recent streaming baselines, and stable 30-minute racing rollouts that preserve scene and subject appearance.

Hugging Face Daily Papers 热度:5 upvotes 作者:Ruoyu Wang, Hao Zhong, Yanming Yang, Jiahao Tian, Lin Liu, Biao Gong, Guoshan Lu, Yiding Tian, Chunhua Shen, Zhenzhong Lan, Chi Zhang

基于本页技术方向关联的整机参数、核心零部件、实测报告与同赛道选型横评。

外部来源

本页事实对应的公开页面,以原文为准。(点击将离开本站)

  1. HF Daily Papers · arXiv 2611.22925:Code World Model 2.0: Towards Infinite Controllable Video Generation ↗访问 2026-11-19