外观
论文HF Daily Papers · arXiv 2610.334134 天前
AV-FABLE基准与NFSR方法突破长视频音视频篡改时序定位
本文由 AI 根据公开内容整理，可能有误，请以原文为准：AV-FABLE: Audio-Visual Manipulation Forensics in Long-Form Videos（HF Daily Papers · arXiv 2610.33413）
核心内容
针对长视频中局部AI篡改事件难以精准定位的问题，研究团队推出AV-FABLE双时间线基准数据集。该数据集包含998小时、16650对原始与篡改孪生视频样本，覆盖27种编辑配置。评测显示零样本AV-LLM检测能力接近随机，但基于冻结感知特征的监督读出可恢复强取证信号，揭示了模型内部表征与语言输出间的语义鸿沟。
基于上述发现，团队提出NFSR（Native Forensics via Separated Readouts）架构，利用独立的音频与视觉时序解码器读取冻结AV-LLM特征，避免联合时间线评估掩盖模态特异性错误。实验表明NFSR在域内及所有分布外测试集（含外部长视频取证基准）均取得最佳定位精度。相关数据集、代码及模型权重将开源，为具身智能长时序视频理解与安全评测提供关键基础设施。
关键看点
- 双时间线基准：AV-FABLE含16650对样本共998小时，支持音视频独立时序定位，解决联合评估掩盖模态错误问题
- AV-LLM表征特性：零样本提示检测近随机，但冻结感知特征经监督读出可恢复强取证信号，证实证据已编码于表征中
- NFSR架构创新：采用分离音视频时序解码器读取冻结特征，在域内及三个OOD设置下均获最优篡改定位性能
- 开源资产释放：完整数据集、训练代码及模型权重将公开，支撑长视频取证与具身智能安全评测研究
工程与产业观察
该工作揭示AV-LLM虽具备底层取证特征但无法通过自然语言有效表达，需专用解码头激活。这对具身智能长时序任务具有双重启示：一是验证了多模态大模型在复杂时序推理上的表征潜力与输出瓶颈；二是为机器人运行日志的异常检测与安全审计提供了可迁移的细粒度时序定位范式，有助于提升自主系统行为的可解释性与可信度。
英文摘要 (Abstract)
Localized AI manipulations in long-form videos often affect brief events within otherwise authentic content, making video-level classification insufficient for forensic analysis. Existing audio-visual deepfake benchmarks largely focus on short talking-head clips, while long-form temporal forensics remains predominantly visual. We introduce AV-FABLE, a dual-timeline benchmark for sparse manipulations in open-scene long-form audio-visual videos. AV-FABLE contains 16,650 pristine--manipulated twin pairs (998 hours) created with 27 editing configurations, with separate audio and visual timelines that enable modality-specific temporal localization and expose errors hidden by joint-timeline evaluation. On AV-FABLE, zero-shot prompting of audio-visual large language models (AV-LLMs) yields near-chance detection and poor temporal localization. In contrast, supervised readouts of their frozen perception features recover strong forensic signals, suggesting manipulation evidence is encoded in these representations but not effectively exposed through language outputs. Motivated by this observation, we propose NFSR(Native Forensics via Separated Readouts), which reads frozen AV-LLM features with separate audio and visual temporal decoders. NFSR achieves the best audio-visual localization performance in-domain and across all three out-of-distribution settings, including transfer to held-out videos from an external long-video forensic benchmark. AV-FABLE, code, and model weights will be publicly released.
Hugging Face Daily Papers 热度：12 upvotes 作者：Yuesheng Huang, Hoi Fan Au, Jiaqi Hou, Ruicheng Zhang, Jiarui He, Haocheng Yang, Bin Hu, Kewei Zhang, Weijian Deng, Jun Zhou, Xiu Li
来源
本页事实对应的公开页面，以原文为准。