Learning speaker-specific characteristics with a deep neural architecture

Ke Chen, Ahmad Salman

    Research output: Contribution to journalArticlepeer-review


    Speech signals convey various yet mixed information ranging from linguistic to speaker-specific information. However, most of acoustic representations characterize all different kinds of information as whole, which could hinder either a speech or a speaker recognition (SR) system from producing a better performance. In this paper, we propose a novel deep neural architecture (DNA) especially for learning speaker-specific characteristics from mel-frequency cepstral coefficients, an acoustic representation commonly used in both speech recognition and SR, which results in a speaker-specific overcomplete representation. In order to learn intrinsic speaker-specific characteristics, we come up with an objective function consisting of contrastive losses in terms of speaker similarity/dissimilarity and data reconstruction losses used as regularization to normalize the interference of non-speaker-related information. Moreover, we employ a hybrid learning strategy for learning parameters of the deep neural networks: i.e., local yet greedy layerwise unsupervised pretraining for initialization and global supervised learning for the ultimate discriminative goal. With four Linguistic Data Consortium (LDC) benchmarks and two non-English corpora, we demonstrate that our overcomplete representation is robust in characterizing various speakers, no matter whether their utterances have been used in training our DNA, and highly insensitive to text and languages spoken. Extensive comparative studies suggest that our approach yields favorite results in speaker verification and segmentation. Finally, we discuss several issues concerning our proposed approach. © 2011 IEEE.
    Original languageEnglish
    Article number6026951
    Pages (from-to)1744-1756
    Number of pages12
    JournalIEEE Transactions on Neural Networks
    Issue number11
    Publication statusPublished - Nov 2011


    • Deep neural architecture
    • hybrid learning strategy
    • overcomplete representation
    • speaker comparison
    • speaker segmentation
    • speaker verification
    • speaker-specific characteristics


    Dive into the research topics of 'Learning speaker-specific characteristics with a deep neural architecture'. Together they form a unique fingerprint.

    Cite this