ArxivWatcher

📡 eess.AS / cs.SD
Audio and Speech Processing, Sound
2026年10月15日
LLM: deepseek-v4-flash
6
论文总数
2
跨领域
0
成功解读
6
待处理
#1
eess.AScs.SD

Energy-Based Models For Speech Synthesis 解读失败跨领域

Wanli Sun, Zehai Tu, Anton Ragni
Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)
Comments: 5 pages, 1 figure, 6 tables. Accepted to ICASSP 2024
查看摘要
Recently there has been a lot of interest in non-autoregressive (non-AR) models for speech synthesis, such as FastSpeech 2 and diffusion models. Unlike AR models, these models do not have autoregressive dependencies among outputs which makes inference efficient. This paper expands the range of available non-AR models with another member called energy-based models (EBMs). The paper describes how noise contrastive estimation, which relies on the comparison between positive and negative samples, can be used to train EBMs. It proposes a number of strategies for generating effective negative samples, including using high-performing AR models. It also describes how sampling from EBMs can be performed using Langevin Markov Chain Monte-Carlo (MCMC). The use of Langevin MCMC enables to draw connections between EBMs and currently popular diffusion models. Experiments on LJSpeech dataset show that the proposed approach offers improvements over Tacotron 2.

📖 深度解读

[LLM 返回为空]

#2
eess.AScs.SD

Score-Based Training for Energy-Based TTS Models 解读失败跨领域

Wanli Sun, Anton Ragni
Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)
Comments: 5 pages, 3 figures, 4 tables. Accepted to Interspeech 2025
查看摘要
Noise contrastive estimation (NCE) is a popular method for training energy-based models (EBM) with intractable normalisation terms. The key idea of NCE is to learn by comparing unnormalised log-likelihoods of the reference and noisy samples, thus avoiding explicitly computing normalisation terms. However, NCE critically relies on the quality of noisy samples. Recently, sliced score matching (SSM) has been popularised by closely related diffusion models (DM). Unlike NCE, SSM learns a gradient of log-likelihood, or score, by learning distribution of its projections on randomly chosen directions. However, both NCE and SSM disregard the form of log-likelihood function, which is problematic given that EBMs and DMs make use of first-order optimisation during inference. This paper proposes a new criterion that learns scores more suitable for first-order schemes. Experiments contrasts these approaches for training EBMs.

📖 深度解读

[LLM 返回为空]

#3
eess.AS

Geometric Alignment without Functional Equivalence: A Layer-wise Analysis of the Speech-Text Modality Gap 解读失败

Ming-Hao Hsu, Xiaohai Tian, Jun Zhang, Lu Lu, Yuxuan Wang, Zhizheng Wu
Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)
Comments: NeurIPS 2026
查看摘要
End-to-end speech large language models (LLMs) often answer the same question less accurately from speech than from text, even when the spoken content matches the text prompt. Existing analyses study this modality gap through the geometry of representations and measure how closely speech representations align with text. Geometric alignment gives the gap a measurable form, but it does not show whether the model uses aligned representations in the same way. This work connects representation geometry with answer formation and asks whether geometric alignment implies functional equivalence. We compare cross-layer Centered Kernel Alignment (CKA) with layer-wise answer margins for four open-weight speech LLMs on SpeechMMLU and VoiceBench BBH. In Qwen2.5-Omni and MiniCPM-o, speech and text representations align in middle layers while their late-layer answer margins still differ, so mid-layer alignment does not imply functional equivalence. Qwen2-Audio instead shows weak alignment, which indicates a different failure mode. An attention intervention at fixed input and token count identifies a causal effect of late attention allocation on answer-code separation, a quantity distinct from answer correctness. Compressing the speech signal or its token sequence repairs a subset of the errors that occur only with speech.

📖 深度解读

[LLM 返回为空]

#4
cs.SD

Omni-Embed-Audio: Leveraging Multimodal LLMs for Robust Audio-Text Retrieval 解读失败

HaeJun Yoo, Yongseop Shin, Insung Lee, Myoung-Wan Koo, Du-Seong Chang
Sound (cs.SD); Computation and Language (cs.CL)
Comments: Accepted at ACL 2026 Main Conference. Final version
查看摘要
Audio-text retrieval systems based on Contrastive Language-Audio Pretraining (CLAP) achieve strong performance on traditional benchmarks; however, these benchmarks rely on caption-style queries that differ substantially from real-world search behavior, limiting their assessment of practical retrieval robustness. We present Omni-Embed-Audio (OEA), a retrieval-oriented encoder leveraging multimodal LLMs with native audio understanding. To systematically evaluate robustness beyond caption-style queries, we introduce User-Intent Queries (UIQs) - five formulations reflecting natural search behaviors: questions, commands, keyword tags, paraphrases, and exclusion-based negative queries. For negative queries, we develop a hard negative mining pipeline and propose discrimination metrics (HNSR, TFR) assessing models' ability to suppress acoustically similar distractors. Experiments on AudioCaps, Clotho, and MECAT show that OEA achieves comparable text-to-audio retrieval performance to state-of-the-art M2D-CLAP, while demonstrating clear advantages in two critical areas: (1) dominant text-to-text retrieval (+22% relative improvement), and (2) substantially superior hard negative discrimination (+4.3%p HNSR@10, +34.7% relative TFR@10), revealing that LLM backbones provide superior semantic understanding of complex queries.

📖 深度解读

[LLM 返回为空]

#5
cs.SD

Remix the Timbre: Diffusion-Based Style Transfer Across Polyphonic Stems 解读失败

Leduo Chen, Junchuan Zhao, Shengchen Li
Sound (cs.SD); Artificial Intelligence (cs.AI)
查看摘要
Timbre transfer aims to modify the timbral identity of a musical recording while preserving its original melody and rhythm. While single-instrument timbre transfer has made substantial progress, existing approaches to multi-instrument settings commonly rely on separate-then-transfer pipelines that propagate source-separation artifacts and process the resulting stems independently. This paper proposes MixtureTT, to the best of our knowledge the first system for flexible per-stem timbre transfer directly from a polyphonic mixture. Given a mixture and a separate timbre reference for each target voice, MixtureTT jointly transfers all stems to the specified instruments. Its Joint Stem Diffusion Transformer first models cross-stem harmonic context and then refines each output under its own content and timbre conditions. All stems are generated within a single joint diffusion process rather than through separate per-stem diffusion runs. This design avoids propagating waveform-level separation errors through a cascade and produces more coherent multi-stem outputs. Despite operating under a harder input condition, MixtureTT outperforms practical separate-then-transfer pipelines on objective and subjective evaluations, while remaining competitive with single-instrument methods given ground-truth isolated stems. Results on both synthesized and real-audio SATB benchmarks further demonstrate its generalization. Controlled ablations show that jointly modeling cross-stem dependencies improves over an equivalent independent-stem variant, while the structure and timing of this coordination are also critical for effective mixture-level timbre transfer.

📖 深度解读

[LLM 返回为空]

#6
cs.SD

C2GA: A Class-Controllable Generative Augmentation Framework for Respiratory Sound Classification 解读失败

Ziqi Ma, Mengyu Han, Anteng Cai, Zhanchong Liu, Bowen Feng, Hang Yu, Sheng Hu
Sound (cs.SD)
Comments: The submitter requests withdrawal due to unresolved authorship and contribution-attribution issues following changes in the original inter-institutional collaboration. The attribution of research contributions requires further clarification. Withdrawal is requested pending resolution of these matters
查看摘要
Background: Respiratory sound classification plays a critical role in the clinical identification of pulmonary pathologies. However, its performance is often hindered by the limited size, severe noise, and class imbalance of real-world auscultation datasets. Although conventional audio augmentation techniques are easy to implement, they may inadvertently distort subtle pathological characteristics. Meanwhile, existing Variational Autoencoder (VAE)- or Generative Adversarial Network (GAN)-based generative approaches often suffer from limited sample fidelity and insufficient controllability over class semantics, particularly under conditions of scarce supervision. Methods: To overcome these limitations, we propose C2GA, a class-controllable generative augmentation framework. C2GA first constructs a semantically rich discrete latent space using a conditional Vector-Quantized Variational Autoencoder (VQ-VAE), in which local acoustic tokens are explicitly decoupled from global class prototypes. Subsequently, a Transformer-based autoregressive prior is trained to generate label-consistent token sequences. These generated tokens are then fused with the corresponding class prototypes and decoded into high-fidelity Mel-spectrograms for data augmentation. Conclusion: These results indicate that C2GA provides an effective and semantically reliable augmentation strategy for respiratory sound analysis. By enabling controllable and high-quality data generation, the proposed framework offers a promising solution for improving the robustness and generalization of respiratory sound classification in realistic clinical scenarios.

📖 深度解读

[LLM 返回为空]