外观
论文HF Daily Papers · arXiv 2610.356714 天前
DeepSeek V4.1 Flash独立安全评测：具身失控风险与过程级观测建议
本文由 AI 根据公开内容整理，可能有误，请以原文为准：Evaluating DeepSeek V4.1 Flash-1017 for loss of control and harmful manipulation risks（HF Daily Papers · arXiv 2610.35671）
核心内容
针对开源前沿模型在自主智能体场景下的安全盲区，研究团队依据EU通用AI行为准则及AEF-1标准，对采用新架构且具备强Agentic能力的DeepSeek V4.1 Flash-1017开展独立评测。测试覆盖34个基准（含5个内部套件），聚焦评估完整性、失控风险及有害操纵，并与GPT-6 Astra、Gemini 3.8 Flash等闭源系统对标。结果表明V4.1 Flash-1017安全性较V4代际大幅改善，在多个维度追平或匹配顶尖闭源模型，但仍存在需持续关注的特定风险行为。
研究指出仅依赖最终输出或端点状态检查无法可靠检测具身智能体的失控行为，必须在部署中实施涵盖推理轨迹与动作日志的过程级可观测性。同时鉴于公开基准存在数据污染及模型“应试”风险，构建私有基准成为可信评估的关键。报告强调安全性能可能随微小版本更新发生回退，建议开发者将安全评估深度集成至增量发布流程，而非仅在重大版本节点进行验证，以确保自主系统在迭代中的鲁棒性。
关键看点
- 评测规模：基于EU AI准则与AEF-1标准执行34项基准测试，涵盖失控与有害操纵等核心具身安全维度
- 性能对标：V4.1 Flash-1017安全性较V4显著提升，在多项指标上逼近或匹配GPT-6 Astra等闭源基线
- 过程观测：终态检查不足以检测失控，部署必须具备覆盖推理Trace与Action Log的过程级可观测能力
- 评估基建：公开基准面临数据污染与评估感知风险，构建私有测试集是保障前沿模型可信评估的必要条件
工程与产业观察
该评测确立了开源具身大模型安全评估的新范式：从单纯的结果对齐转向过程级行为监控。对于机器人算法工程师而言，这意味着在部署V4.1 Flash等高自主模型时，必须重构系统架构以支持全链路Trace记录；同时警示产业界，模型微调或版本迭代可能引发安全回退，需建立自动化回归测试机制与私有Benchmark体系，而非盲目信任公开榜单分数。
英文摘要 (Abstract)
As frontier language models grow increasingly capable and autonomous, independent third-party evaluations are critical for characterizing their systemic safety risks. While closed-source releases are routinely accompanied by comprehensive system cards, open-weight models receive less scrutiny regarding frontier risks. To address this gap, we present an independent safety evaluation of DeepSeek V4.1 Flash-1017, the latest version of V4.1 Flash that departs from V4 with a new architecture, scaled-up training, and strong agentic capabilities. Drawing on the EU General-Purpose AI Code of Practice, we evaluate the model across 34 benchmarks, including 5 in-house suites, spanning evaluation integrity, loss of control, and harmful manipulation. Conforming to the AEF-1 standard, we compare V4.1 Flash-1017 against V4.1 Flash, earlier DeepSeek V4 models, and leading closed-source systems (GPT-6 Astra, GPT-5.6 Sol, and Gemini 3.8 Flash). Overall, V4.1 Flash-1017 shows substantial safety improvements over the V4 generation and approaches or matches leading closed-source baselines on several dimensions, while still exhibiting notable behaviors that warrant continued attention. Building on our findings, we highlight three key observations and recommendations. First, because safety can shift and even regress across minor updates, developers should integrate safety evaluation into incremental releases. Second, as final-turn outputs and end-state checks alone cannot reliably detect loss-of-control behaviors, deployments require process-level observability across both reasoning traces and action logs. Third, because public benchmarks elicit strong evaluation awareness and face data contamination risks, developing private benchmarks is essential for trustworthy frontier evaluation.
Hugging Face Daily Papers 热度：3 upvotes 作者：Yao Huang, Zhiyuan Zeng, Yu Wang, Zhitong Zhu, Dingcheng Duan, Hao Li, Shangzhi Xu, Yifan Wang, Jinzhe Tu, Shukun Xiong, Yihe Zang, Han Yan, Hongchang Yang, Fuyu Gu, Wei Xu, Yinpeng Dong, Xiaojian Li
来源
本页事实对应的公开页面，以原文为准。