IIT Madras is a public technical and research university located in Chennai, Tamil Nadu. Founded in 1959, it is recognised as an Institute of National Importance.

IIT Madras has been ranked as the top engineering institute in India for four years in a row by the National Institutional Ranking Framework of the MHRD

It currently offers undergraduate, postgraduate and research degrees across 16 disciplines in Engineering, Sciences, Humanities and Management. About 596 faculty belonging to science and engineering departments and centres of the Institute are engaged in teaching, research and industrial consultancy.

IIT Madras

The interest in this paper is in efficient configuration of automatic speech recognition (ASR) systems for use by under-served speaker populations. A task domain involving Indian farmers accessing information on agricultural commodities through a spoken dialog system in multiple languages is presented. To facilitate the development of ASR system for this domain, a speech corpus was collected in rural areas from speakers of four languages over wireless cellular channels. This paper investigates the problem of ASR acoustic modelling for this task domain. Continuous density hidden Markov model (CDHMM) and subspace Gaussian mixture model (SGMM) [1] based techniques are used to train acoustic models in four languages: Assamese, Bengali, Hindi and Marathi. Issues relating to limited linguistic resources with their impact on ASR word accuracy for these languages are addressed. © 2012 IEEE.

Department of Electrical Engineering

2012 11th International Conference on Information Science, Signal Processing and their Applications, ISSPA 2012

Subspace based for Indian languages

In this paper, we propose a method to analytically obtain a linear-transformation on the conventional Mel frequency cepstral coefficients (MFCC) features that corresponds to conventional vocal tract length normalization (VTLN)-warped MFCC features, thereby simplifying the VTLN processing. There have been many attempts to obtain such a linear- transformation, but all the previously proposed approaches either modify the signal processing (and therefore not conventional MFCC), or the linear-transformation does not correspond to conventional VTLN-warping, or the matrices being estimated and are data dependent. In short, the conventional VTLN part of an automatic speech recognition (ASR) system cannot be simply replaced with any of the previously proposed methods. Umesh proposed the idea to use band-limited interpolation for performing VTLN-warping on MFCC using plain cepstra. Motivated from this work, Panchapagesan and Alwan proposed a linear-transformation to perform VTLN-warping on conventional MFCC. However, in their approach, VTLN warping is specified in the Mel-frequency domain and is not equivalent to conventional VTLN. In this paper, we present an approach which also draws inspiration from the work of Umesh , and which we believe for the first time performs conventional VTLN as a linear-transformation on conventional MFCC using the ideas of band-limited interpolation. Deriving such a linear-transformation to perform VTLN, would allow us to use the VTLN-matrices in transform-based adaptation framework with its associated advantages and yet would require the estimation of a single parameter. Using four different tasks, we show that our proposed approach has almost identical recognition performance to conventional VTLN on both clean and noisy speech data. © 2012 IEEE.

IEEE Transactions on Audio, Speech and Language Processing

VTLN using analytically determined linear-transformation on conventional MFCC

In this paper, we propose a method to compensate for noise and speaker-variability directly in the Log filter-bank (FB) domain, so that MFCC features are robust to noise and speaker-variations. For noise-compensation, we use Vector Taylor Series (VTS) approach in the Log FB domain, and speaker-normalization is also done in the Log FB domain using Linear Vocal tract length (VTLN) matrices. For VTLN, optimal selection of warp-factor is done in Log FB domain using canonical GMM model, avoiding the two-pass approach needed by a HMM model. Further, this can be efficiently implemented using sufficient statistics obtained from the GMM and the FB-VTLN-matrices. The warp-factor selection using GMM can also be done in cepstral domain by applying DCT matrices without the usual approximations associated with conventional linear-VTLN. The elegance of the proposed approach is that given the speech data, we obtain directly MFCC features that are robust to noise and speaker-variations. The proposed approach, show a significant relative improvement of 31% over baseline on Aurora-4 task. © 2012 IEEE.

GENERAL

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Noise and speaker compensation in the log filter bank domain

This paper describes a novel modification of Histogram Equalization (HEQ) approach to robust speech recognition. We propose separate equalization of the high frequency (HF) and low frequency (LF) bands. We study different combinations of the sub-band equalization and obtain best results when we perform a two-stage equalization. First, conventional HEQ is performed on the cepstral features, which does not completely equalize HF and LF bands, even though the overall histogram equalization is good. In the second stage, an equalization is done separately on the HF and the LF components of the above equalized cepstra. We refer to this approach as Sub-band Histogram Equalization (S-HEQ). The new set of features has better equalization of the sub-bands as well as the overall cepstral histogram. Recognition results show a relative improvement of 12% and 15% over conventional HEQ in WER on Aurora-2 and Aurora-4 databases respectively. Copyright © 2011 ISCA.

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

Sub-band level Histogram Equalization for robust speech recognition

In this paper, we give an overview of the problem of inter-speaker variability and its study in many diverse areas of speech signal processing. We first give an overview of vowel-normalization studies that minimize variations in the acoustic representation of vowel realizations by different speakers. We then describe the universal-warping approach to speaker normalization which unifies many of the vowel normalization approaches and also shows the relation between speech production, perception and auditory processing. We then address the problem of inter-speaker variability in automatic speech recognition (ASR) and describe techniques that are used to reduce these effects and thereby improve the performance of speaker-independent ASR systems. © 2011 Indian Academy of Sciences.

Fulltext

Sadhana - Academy Proceedings in Engineering Sciences

Studies on inter-speaker variability in speech and its application in automatic speech recognition

Recently, Multiple Background Models (M-BMs) [1, 2] have been shown to be useful in speaker verification, where the M-BMs are formed based on different Vocal Tract Lengths (VTLs) among the population. The speaker models are adapted from the particular Background Model (BM) corresponding to their VTL. During test, log likelihood ratio of the test utterance is calculated between claimant model and the corresponding BM. In this paper, instead of using different BM for different speaker, we propose the use of single gender, channel and VTL independent UBM (root-UBM) using the concept of VTL dependent mapping function. The proposed concept is inspired by Feature Mapping (FM) technique used in speaker verification to overcome channel variability. In our proposed method, VTL specific gender independent Gaussian Mixture models (GMMs) are derived from the root-UBM using Maximum a posteriori (MAP) adaptation. The mapping relation is then learned between the root-UBM and the VTL-specific GMM. During training and testing phase, feature vectors are mapped into root-UBM using the best VTL specific model. Then speaker models are adapted from the root-UBM using mapped features. During test, the log likelihood ratio is calculated between target model and root-UBM. Therefore, unlike M-BM system, there is no need to switch to different BMs depending on the claimant. Another advantage of the proposed method is that other additional normalization/compensation techniques can be easily applied since it is in a single UBM frame-work. The experiments are performed on NIST 2004 SRE core condition, and we show that the performance of the proposed method is close to the M-BM system with and without score normalization. © 2011 IEEE.

Use of VTL-wise models in feature-mapping framework to achieve performance of multiple-background models in speaker verification

In this paper, we describe a computationally efficient approach for combining speaker and noise normalization techniques. In particular, we combine the simple yet effective Histogram Equalization (HEQ) for noise compensation with Vocal-tract length normalization (VTLN) for speaker-normalization. While it is intuitive to remove noise first and then perform VTLN, this is difficult since HEQ performs noise compensation in the cepstral domain, while VTLN involves warping in spectral domain. In this paper, we investigate the use of the recently proposed T-VTLN approach to speaker normalization where matrix transformations are directly applied on cepstral features. We show that the speaker-specific warp-factors estimated even from noisy speech using this approach closely match those from clean-speech. Further, using sub-band HEQ (S-HEQ) and TVTLN we get a significant relative improvement of 20% and an impressive 33.54% over baseline in recognition accuracy for Aurora-2 and Aurora-4 task respectively. Copyright © 2011 ISCA.

Efficient speaker and noise normalization for robust speech recognition

In this paper, we propose an anchor modeling scheme where instead of conventional "anchor" speakers, we use eigenvectors that span the Eigen-voice space. The computational advantage of conventional Anchor-modeling based speaker identification system comes from representing all speakers in a space spanned by a small number of anchor speakers instead of having separate speaker models. The conventional "anchor" speakers are usually chosen using data-driven clustering and the number of such speakers are also empirically determined. The use of proposed eigen-voice based anchors provide a more systematic way of spanning the speaker-space and in determining the optimal number of anchors. In our proposed method, the eigenvector space is built using the Maximum Likelihood Linear Regression (MLLR) super-vectors of non-target speakers. Further, the proposed method does not require calculation of the likelihood with respect to anchor speaker models to create the speaker-characterization vector as done in conventional anchor systems. Instead, speakers are characterized with respect to eigen-space by projecting the speaker's MLLR-super vector onto the eigen-voice space. This makes the method computationally efficient. Experimental results show that the proposed method consistently performs better than conventional anchor modeling technique for different number of anchor speakers. Copyright © 2011 ISCA.

Eigen-voice based anchor modeling system for speaker identification using MLLR super-vector

This work deals with strategies to jointly reduce the speaker and environment mismatches in Automatic Speech Recognition. The consequences of environmental mismatch in the performance of conventional Vocal Tract Length Normalization algorithm are analyzed, observing the sensitivity of the warping factor distributions to the SNR fall. A new combined speaker-noise normalization strategy which reduces the effect of noise in VTLN by applying Histogram Equalization is proposed and experimented in AURORA2 and AURORA4 databases. Solid results are obtained and discussed to analyze the effectiveness of the described technique. © 2011 IEEE.

Combining speaker and noise feature normalization techniques for automatic speech recognition

In conventional Speaker-Identification using GMM-UBM framework, the likelihood of the given test utterance is computed with respect to all speaker-models before identifying the speaker, based on the maximum likelihood criterion. The calculation of likelihood score of the test utterance is computationally intensive, especially when there are tens of thousands of speakers in database. In this paper, we propose a computationally efficient (Fast) method to calculate the likelihood of the test utterance using speaker-specific Maximum Likelihood Linear Regression (MLLR) matrices (which are precomputed) and sufficient statistics estimated from the test utterance only once. We show that while this method is an order of magnitude faster, there is some degradation in performance. Therefore, we propose a cascaded system with the Fast MLLR system identifying the top-N most probable speakers, followed by a conventional GMM-UBM to identify the most probable speaker from the top-N speakers. Experiments performed on the NIST 2004 database indicate that the cascaded system provides a speed up of 3.16 and 6.08 times for 1-side test (core condition) and 10 sec. test condition respectively, with a marginal degradation in accuracy over the conventional GMM-UBM system. © Odyssey 2010: Speaker and Language Recognition Workshop. All rights reserved.

Odyssey 2010: Speaker and Language Recognition Workshop

Computationally efficient speaker identification for large population tasks using MLLR and sufficient statistics

In speaker identification, most of the computational processing time is required to calculate the likelihood of the test utterance of the unknown speaker with respect to the speaker models in the database. When number of speakers in the database is in the order of 10,000 or more, then computational complexity becomes very high. In this paper, we propose a Maximum Likelihood Linear Regression (MLLR) based fast method to calculate the likelihood from the speaker model using the MLLR matrix. The proposed technique will help to quickly find the best N speakers during identification. After that final speaker identification task can be done within the N selected speakers using any conventional method of speaker identification. The comparative study of the proposed method is done in terms of processing time with the state-of-the-art GMM-UBM based system on NIST 2004 SRE. The proposed technique performs faster than GMM-UBM based system with some degradation in system accuracy. ©2010 IEEE.

Proceedings of 16th National Conference on Communications, NCC 2010

Fast approach to speaker identification for large population using MLLR and sufficient statistics

Anchor modeling technique has been shown to be useful in reducing computational complexity for speaker identification and indexing of large audio database. In this technique, speakers are projected onto a talker space spanned by a set of predefined anchor models which are usually represented by Gaussian Mixture Models (GMMs). The characterization of each speaker involves calculation of likelihood with each of the anchor models, and is therefore expensive even in the GMM Universal Background model (GMM-UBM) frame work using top-C mixtures per feature vector. In this paper, we propose a computationally efficient (Fast) method to calculate the likelihood of the speech utterances using anchor speaker-specific Maximum Likelihood Linear Regression (MLLR) matrices and sufficient statistics estimated from the utterance. We show that the proposed method is faster by an order of magnitude for evaluating the speaker characterization vector. Since anchor models use simple distance measures to identify speakers, they are used as a first stage to select N probable speakers and then cascaded by a conventional GMM-UBM stage which finally identifies the speaker from this reduced set. We show that the proposed method in cascade combination perform 4.21× faster than the conventional cascade anchor model system with comparable performance. The experiments are performed on NIST 2004 SRE in core condition. © 2010 ISCA.

Proceedings of the 11th Annual Conference of the International Speech Communication Association, INTERSPEECH 2010

Fast computation of speaker characterization vector using MLLR and sufficient statistics in anchor model framework

It is common to use a single speaker independent large Gaussian Mixture Model based Universal Background Model (GMM-UBM) as the alternative hypothesis for speaker verification tasks. The speaker models are themselves derived from the UBM using Maximum a Posteriori (MAP) adaptation technique. During verification, log likelihood ratio is calculated between the target model and the GMM-UBM to accept or reject the claimant. The use of a single UBM for different groups of population may not be appropriate especially when the impostors are close to the target speaker. In this paper, we investigate the use of Speaker Cluster-wise UBM (SC-UBM) for a group of target speakers based on two different similarity measures. In the first approach, speakers are grouped into different clusters depending on their Vocal Tract Lengths (VTLs). The group of speakers having same VTL parameter indicates similarity in vocal-tract geometry and constitutes a speaker-dependent characteristic. In the second approach, we use Maximum Likelihood Linear Regression (MLLR) matrices of target speakers to create MLLR super-vectors and use them to cluster speakers into different groups. The SC-UBMs are derived from GMM-UBM using MLLR adaptation using data from the corresponding group of target speakers. Finally, speaker dependent models are adapted from their respective SC-UBM using MAP. In the proposed method, log likelihood ratio is calculated between target model and its corresponding SC-UBM. We compare performance of the above method with the single UBM method for varying number of clusters. The experiments are performed on the NIST 2004 SRE core condition and we show that the proposed method with a slight increase in the number of UBMs always outperforms the conventional single GMM-UBM system. © Odyssey 2010: Speaker and Language Recognition Workshop. All rights reserved.

Investigation of speaker-clustered UBMs based on vocal tract lengths and MLLR matrices for speaker verification

In speaker verification task requires some sort of background model for the system to make decision. Most of the cases, a speaker independent large Gaussian Universal Background Model (GMM-UBM) is used. In this paper, we propose to use a Speaker Cluster-wise UBM (SC-UBM) for a group of target speakers. In this method, the target speakers are clustered into group based on their similarity in Vocal Tract Length Normalization (VTLN) parameter. The VTLN parameter depends on the physiological structure of human speech production system. Hence, the group of speakers with same VTLN factor represent a speaker with unique characteristic. The SC-UBMs are derived from GMM-UBM with Maximum Likelihood Linear Regression (MLLR) by pooling data from the specific group of target speakers. The speaker dependent models are then adapted from their respective SC-UBM using Maximum a Posteriori (MAP) method. During verification, the log likelihood ratio for the claimant is calculated with respect to the corresponding group specific UBM. The comparative study are performed on NIST 2004 SRE in core condition. The SCUBM system reduced equal error rate (EER) by 9% over the GMM-UBM system. ©2010 IEEE.

Vocal tract length normalization factor based speaker-cluster UBM for speaker verification

In this paper we study the effect of use of jacobian in different linear transformation (LT) based methods of VTLN. In conventional VTLN, the jacobian is highly non-linear and can not be computed and hence is ignored. In the LT based VTLN, since VTLN scaling is expressed as a matrix multiplication of un-warped MFCC features, jacobian is simply turns out as the determinant of the VTLN warp matrices. Hence in this framework of VTLN it is possible to account for jacobian. Two different methods, namely, L-VTLN and T-VTLN, are used for implementing LT based VTLN. By conducting experiments on the RM task and the TIDIGITs databases in matched and mismatched speaker conditions, the performance of using jacobian in warp-factor estimation have been evaluated. It is observed that in almost every matched and mis-matched speaker conditions jacobian improves performance in L-VTLN framework. In TVTLN, however, jacobian does not improve the performance in any mis-matched speaker conditions. The cases in which jacobian degrades performance in L-VTLN and T-VTLN have been studied in detail. ©2010 IEEE.

Effect of jacobian compensation in linear transformation based VTLN under matched and mis-matched speaker conditions

In this paper, we study the use of different frequency warpfactors for different acoustic classes in a computationally efficient frame-work of Vocal Tract Length Normalization (VTLN). This is motivated by the fact that all acoustic classes do not exhibit similar spectral variations as a result of physiological differences in vocal tract, and therefore, the use of a single frequency-warp for the entire utterance may not be appropriate. We have recently proposed a VTLN method that implements VTLN-warping through a linear-transformation (LT) of the conventional MFCC features and efficiently estimates the warp-factor using the same sufficient statistics as that are used in CMLLR adaptation. In this paper we have shown that, in this framework of VTLN, and using the idea of regression class tree, we can obtain separate VTLN-warping for different acoustic classes. The use of regression class tree ensures that warp-factor is estimated for each class even when there is very little data available for that class. The acoustic classes, in general, can be any collection of the Gaussian components in the acoustic model. We have built acoustic classes by using data-driven approach and by using phonetic knowledge. Using WSJ database we have shown the recognition performance of the proposed acoustic class specific warp-factor both for the data driven and the phonetic knowledge based regression class tree definitions and compare it with the case of the single warp-factor. Copyright © 2009 ISCA.

World University root node

Acoustic class specific VTLN-warping using regression class trees

In this paper, we propose to combine the rapid adaptation capability of conventional Vocal Tract Length Normalization (VTLN) with the computational efficiency of transform-based adaptation such as MLLR or CMLLR. VTLN requires the estimation of only one parameter and is, therefore, most suited for the cases where there is little adaptation data (i.e. rapid adaptation). In contrast, transform-based adaptation methods require the estimation of matrices. However, the drawback of conventional VTLN is that it is computationally expensive since it requires multiple spectral-warping to generate VTLN-warped features. We have recently shown that VTLN-warping can be implemented by a linear-transformation (LT) of the conventional MFCC features. These LTs are analytically pre-computed and stored. In this frame-work of LT VTLN, computational complexity of VTLN is similar to transform-based adaptation since warp-factor estimation can be done using the same sufficient statistics as that are used in CMLLR. We show that VTLN provides significant improvement in performance when there is small adaptation data as compared to transform-based adaptation methods. We also show that the use of an additional decorrelating transform, MLLT, along with the VTLN-matrices, gives performance that is better than MLLR and comparable to SAT with MLLT even for large adaptation data. Further we show that in the mismatched train and test case (i.e. poor first-pass transcription), VTLN provides significant improvement over the transform-based adaptation methods. We compare the performances of different methods on the WSJ, the RM and the TIDIGITS databases. Copyright © 2009 ISCA.

Department of Mechanical Engineering

Using VTLN matrices for rapid and computationally-efficient speaker adaptation with robustness to first-pass transcription errors

The performance of conventional VTLN for mis-matched train and test speaker conditions (e.g. adult-train child-test) does not approach the performance of matched speaker conditions (e.g. child-train child-test). In this paper, we investigate this problem and propose methods to reduce this gap in performance. We use our recently proposed linear transformation approach to VTLN, that also enables us to study the effect of Jacobian unlike conventional VTLN. The main advantage of transform-based VTLN over adaptation based approaches (like CMLLR), is that it does not require any matrix estimation. We argue that the degraded VTLN performance under mismatched speaker conditions is due to the significant frequency warping that is necessary for normalization which leads to a mismatch between the correlation in the feature components of the test data and the covariance structure of the trained/normalized model. We show that the use of a global de-correlating transform (MLLT) leads to improved VTLN performance. We finally show that using both Jacobian and MLLT together improves the VTLN performance for mis-matched cases with the performance approaching that of matched speaker conditions. ©2009 IEEE.

Improving the performance of VTLN under mismatched speaker conditions and making it approach that of matched speaker conditions

In this paper, we present a study to understand the relation among spectra of speakers enunciating the same sound and investigate the issue of uniform versus non-uniform scaling. There is a lot of interest in understanding this relation as speaker variability is a major source of concern in many applications including Automatic Speech Recognition (ASR). Using dynamic programming, we find mapping relations between smoothed spectral envelopes of speakers enunciating the same sound and show that these relations are not linear but have a consistent non-uniform behavior. This non-uniform behavior is also shown to vary across vowels. Through a series of experiments, we show that using the observed non-uniform relation provides better vowel normalization than just a simple linear scaling relation. All results in this paper are based on vowel data from TIMIT, Hillenbrand et al. and North Texas databases. Copyright © 2009 ISCA.

Department Of Physics

Characterizing speaker variability using spectral envelopes of vowel sounds

In this paper, we propose to use Vocal Tract Length Normalization (VTLN) to build the Universal Background Model (UBM) for a closed set speaker identification system. Vocal Tract Length (VTL) differences among speakers is a major source of variability in the speech signal. Since the UBM model is trained using data from many speakers, it statistically captures this inherent variation in the speech signal, which results in a "coarse" model in the acoustic space. This may cause the adapted speaker models obtained from the UBM model to have significantly high overlap in the acoustic space. We hypothesize that the use of VTLN will help in compacting the UBM model and thus the speaker adapted models obtained from this compact model will have better speaker-separability in the acoustic space. We perform experiments on MIT, TIMIT and NIST 2004 SRE databases and show that using VTLN we can achieve lesser Identification Error Rates as compared to the conventional GMM-UBM based method. Copyright © 2009 ISCA.