外观
论文arXiv 2610.1971614 天前
Soloist：用于野外盲语音分离的生成模型
原文链接：Soloist: A Generative Model for Blind Voice Separation in the Wild（arXiv 2610.19716 · 发布日期：2026-10-14）
Soloist: A Generative Model for Blind Voice Separation in the Wild
Soloist面向野外盲语音分离，用条件生成音频码序列的方法，在合成混合与真实录音上验证。
核心要点
- 现实语音分离需从重叠语音、噪声和音乐中恢复个体声音，但孤立源监督稀缺。
- 模型按自回归方式生成个体声音频码流，条件为混合音频，无需目标说话人查询或预设人数。
- 两阶段训练先结合合成监督与真实录音时间线预测，再用模型裁判奖励在真实录音上强化学习。
- 实验显示其可分离多说话人重叠和噪声录音，并从音乐伴奏中提取歌声，且用于全双工对话流程。
验证范围与局限：验证范围包括合成混合、真实录音、多说话人重叠、噪声录音、音乐伴奏和真实对话。
英文摘要 (Abstract)
Real-world voice separation must recover individual voices from overlapping speech, environmental noise, and music, yet isolated source recordings are rarely available for supervision. We introduce Soloist, a generative model for blind voice separation that autoregressively generates individual voice streams as audio-codec sequences conditioned on the mixture, without requiring a target-speaker query or a predefined voice count. We propose a two-stage training paradigm to learn from synthetic mixtures and adapt to real recordings. The first stage combines supervised separation on synthetic mixtures with isolated source targets and an auxiliary task that predicts speaker timelines from real recordings, learning who speaks when from natural acoustics. The second stage applies reinforcement learning to real recordings, with rewards supplied by a model judge that compares candidate separations based on voice purity and content preservation without isolated source targets. Across diverse scenarios, Soloist recovers clean speech from multi-speaker overlap and noisy recordings, and extracts singing vocals from music accompaniment while maintaining relatively high pitch accuracy and intelligibility. On real-world recordings with unknown voice counts, Soloist also separates voices well. Finally, we integrate Soloist into a full-duplex conversation processing pipeline that assembles separated outputs into continuous speaker tracks, demonstrating its practical utility in real conversations.
作者：Chen Yang, Chufan Yu, Botian Jiang, Jingqi Chen, Wenxuan Wang, Yitian Gong, Zhaoye Fei, Xipeng Qiu
来源
本页事实对应的公开页面，以原文为准。