论文HF Daily Papers · arXiv 2610.401915 天前

视频生成迈向智能体化：Harness架构支撑世界模型与递归自改进

论文提出Agent Harness运行时架构，将视频生成从端到端合成转向具备规划、工具调用与反馈的智能体模式。该框架支持脚本编排、混合世界建模及递归自改进（RSI），为具身智能提供可交互仿真环境与模型自主迭代新范式。

AI 整理

仿真与训练
具身模型

本文由 AI 根据公开内容整理，可能有误，请以原文为准：Video Generation Meets Agent Harness: Agentic Video Generation, Agentic World Modeling and Recursive Self-Improvement（HF Daily Papers · arXiv 2610.40191）

核心内容

arXiv 2610.40191论文指出视频生成正从端到端合成转向Agentic Generation，核心在于引入Agent Harness作为可执行运行时。该系统负责构建上下文、暴露工具接口、维护状态、路由反馈及管理计算预算，使视频模型具备长程规划、代码交互与多智能体协调能力，解决了传统生成模型缺乏可控性与逻辑一致性的工程痛点。

技术层面提出混合世界模型设计，由游戏引擎或Agent编写程序维护物理状态，视频模型仅负责渲染观测，实现状态与表征解耦。在递归自改进（RSI）机制中，系统不仅修正输出，还更新Harness状态、生成器参数及评估器本身。这种闭环机制要求Agent具备音视频感知、记忆复用及反馈引导控制能力，直接关联具身仿真环境构建与策略训练数据生成。

关键看点

Agent Harness架构：定义包含上下文构建、工具暴露、状态维护及预算管理的运行时，支撑脚本规划与多镜头编辑等复杂任务流
混合世界建模：采用游戏引擎或代码维护世界状态、视频模型渲染观测的解耦设计，解决纯视频模型物理一致性差的问题
递归自改进RSI：建立涵盖输出修正、参数更新、评估器优化及改进程序自身迭代的四层反馈闭环，驱动模型与Harness协同进化
核心Agent能力：明确音视频感知、长程规划、代码交互、技能复用及多智能体协调为支撑自主视频生产与世界模型的关键要素

工程与产业观察

Agent Harness为世界模型提供了工程化落地骨架，其混合架构与RSI机制对具身智能具有双重价值：既可作为高保真Sim-to-Real仿真器降低实机训练成本，又能通过自主迭代生成高质量合成数据。这标志着视频生成从内容创作工具向机器人认知基础设施转型，算法工程师应关注状态管理与反馈回路设计。

英文摘要 (Abstract)

Video generation is shifting from end-to-end synthesis toward agentic generation, where planning, tool use, and feedback enable coherent, controllable multi-shot videos. In parallel, video world models provide action-conditioned observations and predicted futures, serving as interactive environments for agents. We study their intersection through the agent harness: an executable runtime that constructs context, exposes tools, maintains state, routes feedback, and manages budgets around video models. This report examines three directions. First, agentic video generation, where harnesses coordinate scripting, shot planning, asset management, production, and editing, alongside the evaluation challenges of these systems. Second, agentic world modeling, where harnesses mediate observation, control, execution, and state. We discuss hybrid designs in which game engines or agent-written programs maintain world state while video models render observations, and how agents and world models can mutually evaluate and improve one another. Third, recursive self-improvement (RSI) in video model development, distinguishing output revisions, updates to harness state, generator parameters, and evaluators, and changes to the improvement procedure itself. We identify how these improvements feed back into subsequent development. Across these directions, we analyze essential agent capabilities, including audiovisual perception, long-horizon planning, memory, code-based interaction, feedback-guided control, skill reuse, and multi-agent coordination. We conclude with challenges in autonomous video production, lifelong memory, programmable hybrid control, autonomous model development, provenance, cost accounting, and evaluation linking video quality and task performance to sustained improvements in models and harnesses.

Hugging Face Daily Papers 热度：58 upvotes 作者：Fan Liaoyuan, Yuhao Zheng, Dongxing Mao, Hangyu Ran, Siyuan Hu, Weijia Mao, Rui Zhao, Weijia Wu, Guian Fang, Yanzhe Chen, Zeyu Zhu, Yuxin Jiang, Yiren Song, Yuchao Gu, Junchao Yi, Qi Hu, Dan Kong, Ale…