IIT Madras is a public technical and research university located in Chennai, Tamil Nadu. Founded in 1959, it is recognised as an Institute of National Importance.

IIT Madras has been ranked as the top engineering institute in India for four years in a row by the National Institutional Ranking Framework of the MHRD

It currently offers undergraduate, postgraduate and research degrees across 16 disciplines in Engineering, Sciences, Humanities and Management. About 596 faculty belonging to science and engineering departments and centres of the Institute are engaged in teaching, research and industrial consultancy.

IIT Madras

In this paper, we first show that accounting for Jacobian in Vocal-Tract Length Normalization (VTLN) will degrade the performance when there is a mismatch between the train and test speaker conditions. VTLN is implemented using our recently proposed approach of linear transformation of conventional MFCC, i.e. a feature transformation. In this case, Jacobian is simply the determinant of the linear transformation. Feature transformation is equivalent to the means and covariances of the model being transformed by the inverse transformation while leaving the data unchanged. Using a set of adaptation experiments, we analyze the reasons for the degradation during Jacobian compensation and conclude that applying the same VTLN transformation on both means and variances does not fully match the data when there is a mismatch in the speaker conditions. This may have similar implications for constrained-MLLR in mismatched speaker conditions. We then propose to use covariance adaptation on top of VTLN to account for the covariance mismatch between the train and the test speakers and show that accounting for Jacobian after covariance adaptation improves the performance. Copyright © 2009 ISCA.

Department of Electrical Engineering

World University root node

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

A study on the influence of covariance adaptation on Jacobian compensation in vocal tract length normalization

In this paper, a well-motivated nonuniform speaker normalization model that affinely relates the formant frequencies of speakers enunciating the same sound is proposed. Using the proposed affine model, the corresponding universal-warping function that is required for normalization is shown to have the same parametric form as the mel scale formula. The parameters of this universal-warping function are estimated from the vowel formant data and are shown to be close to the commonly used formula for the mel scale. This shows an interesting connection between nonuniform speaker normalization and the psychoacoustics based mel scale. In addition, the affine model fits the vowel formant data better than commonly used ad hoc normalization models. This work is motivated by a desire to improve the performance of speaker-independent speech recognition systems, where speaker normalization is conventionally done by assuming a linear-scaling relationship between spectra of speakers. The proposed affine relation is extended to describe the relationship between spectra of speakers enunciating the same sound. On a telephone-based connected digit recognition task, the proposed model provides improved recognition performance over the linear-scaling model.

The Journal of the Acoustical Society of America

Nonuniform speaker normalization using affine transformation

In this paper, we present a linear transformation (LT) to obtain warped features from unwarped features during vocal-tract length normalisation (VTLN). This LT between the warped and unwarped features is obtained within the conventional MFCC framework without any modification in the signal processing steps involved during the feature extraction stage. Further using the proposed LT, we study the effect of the Jacobian on the VTLN performance and show that it provides additional improvement in the recognition performance. The Jacobian of the proposed LT is simply the determinant of the LT matrix. Jacobian compensation is not done in conventional VTLN as the relation between warped and unwarped features is not known. We also study the effect of cepstral variance normalisation (CVN), which is often used as an approximation for Jacobian compensation in conventional VTLN. We show that the proposed Jacobian compensation gives better or comparable performance when compared to CVN. Copyright © 2008 ISCA.

Study of Jacobian compensation using linear transformation of conventional MFCC for VTLN

In this paper, we present an approach to generate speaker invariant features for automatic speech recognition (ASR) using the idea of spectral centre of gravity(CG). This is based on the observation that if two signals are delayed versions of one another, then their CG's also differ by the same amount. We exploit this idea to appropriately shift the mel warped log compressed spectra using the estimated CG to obtain speaker invariant features. The use of such speaker invariant or normalised features helps improve the recognition performance of speaker-independent ASR. We show that our proposed approach is computationally efficient when compared to a commonly used method of normalisation called Vocal Tract Length Normalisation (VTLN). We present normalisation results to show that the performance of our proposed approach is comparable to conventional VTLN and yet has the advantage of computational efficiency. Copyright © 2008 ISCA.

Department of Mathematics

Use of spectral centre of gravity for generating speaker invariant features for automatic speech recognition

In this paper, we develop a computationally efficient approach for warp factor estimation in Vocal Tract Length Normalization (VTLN). Recently we have shown that warped features can be obtained by a linear transformation of the unwarped features. Using the warp matrices we show that warp factor estimation can be efficiently performed in an EM framework. This can be done by collecting Sufficient Statistics by aligning the unwarped utterances only once. The likelihood of warped features, which are necessary for warp factor estimation, are computed by appropriately modifying the sufficient statistics using the warp matrices. We show using OGI, TIDIGITS and RM task that this approach has recognition performance that is comparable to conventional VTLN and yet is computationally more efficient. Copyright © 2008 ISCA.

A computationally efficient approach to warp factor estimation in VTLN using EM algorithm and sufficient statistics

In this work, we present a speaker-normalization method based on the idea that the speaker-dependent scale-factor can be separated out as a fixed translation factor in an alternate domain. We also introduce a non-linear frequency-scaling model motivated by the analysis of speech data. The proposed shift-based normalization approach is implemented using a maximum-likelihood (ML) search for the translation factor in the alternate domain. The advantage of our approach is that we are able to show the relationship between conventional frequency-warping based vocal-tract length normalization (VTLN) methods and the methods based on shifts in psycho-acoustic scale thus providing a unifying frame-work for speaker-normalization. Additionally, in our approach it is simple to show that the shifting required for normalization can be expressed as a linear transformation in the cepstral domain. This is important for computational efficiency since we do not have to recompute the features by re-doing the signal processing for each scale/translation factor as is usually done in conventional normalization. We present recognition results using our proposed approach on a digit recognition task and show that the non-linear scaling model provides relative improvement of 4% for adults and 7.5% for children when compared to the linear-scaling model. © 2007 Elsevier B.V. All rights reserved.

Department of Aerospace Engineering

Speech Communication

A shift-based approach to speaker normalization using non-linear frequency-scaling model

In this paper, we study the effect of filter bank smoothing on the recognition performance of children's speech. Filter bank smoothing of spectra is done during the computation of the Mel filter bank cepstral coefficients (MFCCs). We study the effect of smoothing both for the case when there is vocal-tract length normalization (VTLN) as well as for the case when there is no VTLN. The results from our experiments indicate that unlike conventional VTLN implementation, it is better not to scale the bandwidths of the filters during VTLN - only the filter center frequencies need be scaled. Our interpretation of the above result is that while the formant center frequencies may approximately scale between speakers, the formant bandwidths do not change significantly. Therefore, the scaling of filter bandwidths by a warp-factor during conventional VTLN results in differences in spectral smoothing leading to degradation in recognition performance. Similarly, results from our experiments indicate that for telephone-based speech when there is no normalization it is better to use uniform-bandwidth filters instead of the constant- like filters that are used in the computation of conventional MFCC. Our interpretation is that with constant- filters there is excessive spectral smoothing at higher frequencies which leads to degradation in performance for children's speech. However, the use of constant- filters during VTLN does not create any additional performance degradation. As we will show, during VTLN it is only important that the filter bandwidths are not scaled irrespective of whether we use constant- or uniform-bandwidth filters. With our proposed changes in the filter bank implementation we get comparable performance for adults and about 6% improvement for children both for the case of using VTLN as well as the for the case of not using VTLN on a telephone-based digit recognition task. © 2006 IEEE.

IEEE Transactions on Audio, Speech and Language Processing

A study of filter bank smoothing in MFCC features for recognition of children's speech

We argue that when individuals enunciate sounds which are perceived to be the same, the sounds have the commonalty that their spectra can be transformed into a new domain which results in identical spectra except for a speaker dependent translation factor. We call the transformation function the speech scale. The speech scale is experimentally obtained. In this paper we explore the mathematical issues involved and obtain various criteria for when a transformation to a new domain results in a speaker independent transform.

Noise and Fluctuations in Biological, Biophysical, and Biomedical Systems

Fluctuations in speech

In this paper, we consider the generation of features for automatic speech recognition (ASR) that are robust to speaker-variations. One of the major causes for the degradation in the performance of ASR systems is due to inter-speaker variations. These variations are commonly modeled by a pure scaling relation between spectra of speakers enunciating the same sound. Therefore, current state-of-the art ASR systems overcome this problem of speaker-variability by doing a brute-force search for the optimal scaling parameter. This procedure known as vocal-tract length normalization (VTLN) is computationally intensive. We have recently used Scale-Transform (a variation of Mellin transform) to generate features which are robust to speaker variations without the need to search for the scaling parameter. However, these features have poorer performance due to loss of phase information. In this paper, we propose to use the magnitude of Scale-Transform and a pre-computed "phase"-vector for each phoneme to generate speaker-invariant features. We compare the performance of the proposed features with conventional VTLN on a phoneme recognition task.

IJCAI International Joint Conference on Artificial Intelligence

Speaker-invariant features for automatic speech recognition

In the paper, we present a novel linear transformation approach to frequency warping during vocal tract length normalisation(VTLN) using the idea of dynamic frequency warping(DFW). Linear transformation among the mel-frequency cepstral coefficients (MFCC) provides computational advantage of not having to recompute features for each warp factor in VTLN. The proposed method uses the idea of separating the smoothing and the frequency warping operations in the feature extraction stage unlike the conventional approach where both operations are integrated into the filter-bank operation. The advantage of the proposed DFW approach is that, we can obtain a transformation matrix for any arbitrary warping even when we do not know the functional form or mapping of the warping function. We compare the performance of the proposed method along with approaches proposed in [4] and [5] on one phone classification and two digit recognition tasks.

Linear transformation approach to VTLN using dynamic frequency warping

In this paper, we study non-linear frequency-warping functions that are commonly used in speaker normalization. This study is motivated by our recently proposed affine transformation model for speaker normalization [1] which has provided improved recognition performance when compared to uniform scaling model [1, 2]. In this work, using formant data from Peterson & Barney and Hillenbrand vowel databases, we analyze the behavior of scale factor as a function of frequency. The empirical observation [3, 4] shows that while uniform scaling assumption may be valid at higher frequencies, there are significant deviations at low frequencies. We show that while our recently proposed model has behavior similar to the empirical result, the behavior of many of the commonly used non-linear models (including that of Eide-Gish, power law and bilinear transformation) differ significantly from the empirical result. This difference in behavior from the empirical observation may explain the limited improvement in recognition performance provided by these non-linear models when compared to conventional uniform-scaling model. We also show that our proposed model does better fitting to the formant data than these non-linear models. We, therefore, conclude that the affine-transformation model may be a more appropriate non-linear model for speaker normalization. © 2006 IEEE.

Department of Metallurgical and Materials Engineering

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Study of non-linear frequency warping functions for speaker normalization

In this paper we present an efficient and flexible approach to VTLN warping factor estimation. Due to the equivalence of frequency warping and linear transformation of cepstral coefficients, warping factors can be efficiently estimated by accumulating the sufficient statistics for linear transformation estimation, and searching the constrained space of transformations given by the explicit mapping between warping factors and linear transformation matrices. We show that the positive effect of using a properly normalized optimization criterion for warping factor estimation, which has been previously demonstrated for a signal analysis front-end without a filter-bank, carries over to a MFCC front-end, resulting in a net improvement in word error rate. © 2006 IEEE.

VTLN warping factor estimation using accumulation of sufficient statistics

In this paper, we show that frequency-warping (including VTLN) can be implemented through linear transformation of conventional MFCC. Unlike the Pitz-Ney [1] continuous domain approach, we directly determine the relation between frequency-warping and the linear-transformation in the discrete-domain. The advantage of such an approach is that it can be applied to any frequency-warping and is not limited to cases where an analytical closed-form solution can be found. The proposed method exploits the bandlimited interpolation idea (in the frequency-domain) to do the necessary frequency-warping and yields exact results as long as the cepstral coefficients are que-frency limited. This idea of quefrency-limitedness shows the importance of the filter-bank smoothing of the spectra which has been ignored in [1, 2]. Furthermore, unlike [1], since we operate in the discrete domain, we can also apply the usual discrete-cosine transform (i.e. DCT-II) on the logarithm of the filter-bank output to get conventional MFCC features. Therefore, using our proposed method, we can linearly transform conventional MFCC cepstra to do VTLN and we do not require any recomputation of the warped-features. We provide experimental results in support of this approach.

9th European Conference on Speech Communication and Technology

Implementing frequency-warping and VTLN through linear transformation of conventional MFCC

A fundamental issue of speech is that when different individuals enunciate the same perceived sound, the corresponding spectra are different, but since we perceive them to be the same, there must have a commonality that the ear extracts to recognize the same perceived sound. In previous publications we have established this commonality and have argued that the "spectra of sounds made by different individuals and perceived to be the same can be transformed into each other by a universal warping function". We call the warping function the speech scale and in previous works we have obtained it experimentally from actual speech. In this paper we give the mathematical equation that allows one to obtain the transformation function so that the transformation results in identical warped spectra except for a translation factor.

SPIE Digital Library

The speech scale and spectral transformation

In this paper, we present improvements in the estimation of frequency-dependent scaling function for non-uniform speaker normalization when compared to the method in [1]. Further, unlike [1], using the estimated frequency-dependent scaling function, γ(f), we estimated the universal warping function that is necessary to separate out the speaker-dependent term as a translation factor and show that it is similar to mel-scale. Since, the proposed warping function is similar to mel-scale, we argue that our study "justifies" the usage of mel-scale in speech recognition, not only from the point of view of psychoacoustics but also from the view point of speaker normalization. Finally, in [2], we have assumed the commonly used formula for mel-scale, mel = 2595 log10 1 + f/700 for the universal warping function. In this paper, we have tried to fit a mel-like formula to the estimated universal warping function and use it to do non-uniform speaker normalization. We present the recognition results using these different universal warping functions with word error rate as the performance measure. © 2004 IEEE.

2004 International Conference on Signal Processing and Communications, SPCOM

Non-uniform speaker normalization using frequency-dependent scaling function

Our investigation into the front-end signal processing for maximum likelihood based speaker normalization reveals that in the linear scaling model, it is more appropriate (and evidently more correct) to assume that the spectral envelopes of any two speakers for same sound are linearly scaled versions of one and another, rather than assuming that the whole magnitude spectra (including pitch harmonics) are scaled. The use of the proposed model and its implementation results in about 4% and 7% relative improvement for adults and children respectively on a digit recognition task.

An investigation into front-end signal processing for speaker normalization

Vocal tract length normalisation (VTLN) is a commonly used speaker normalisation approach. It is attractive compared to many normalisation schemes as it is typically dependent on only a single parameter, allowing the warp factors to be robustly calculated on little data. However, the scheme normally requires explicitly coding the data at multiple warp factors. Furthermore, it is only possible to approximate the Jacobian associated with the VTLN transformation. A new, simple, linear approximation to VTLN is described in this paper. This linear approximation allows the Jacobian to be exactly computed. It can also be highly efficient in terms of warp factor estimation and application of the warp factors. Both the linear and standard CUED VTLN schemes were evaluated in the 2003 BNE evaluation framework and found to yield similar performance. When used in system combination both VTLN schemes yielded slight gains over the baseline system.

8th International Conference on Spoken Language Processing, ICSLP 2004

Using VTLN for broadcast news transcription

In this paper, we propose a mathematical model to describe the relation between the formant frequencies of speakers and show that with the proposed affine model, speaker differences separate out as translation factors when a "mel-like" warping is performed. Using speech data we estimate the parameters of this warping function and show that it is close to the usual mel-formula. This model is motivated by Rohit et al.'s shift-based non-uniform speaker-normalization method, which provides improvement over the conventional maximum-likelihood based speaker normalization methods. We therefore provide a unified framework that relates the relationship between formants of speakers and method of removing speakers difference (which involves mel-warping) in a neat mathematical framework which is substantiated by our recognition experiments.

Non-uniform speaker normalization using affine-transformation

In previous works, Umesh et al, demonstrated that phonetically similar vowels spoken by different individuals are related by a simple translation in a universal warped spectral representation. They experimentally derived this function and called it the "speech-scale". We present further experimental evidence, based on a large data set, validating the speech-scale. We also estimate speaker-specific scale factors based on the speech-scale, and we present a vowel classification experiment, which demonstrates a significant performance improvement through a normalization based on the speech-scale. The results we present are based on formant estimates of vowels in a Western Michigan vowel database.

Proceedings of SPIE - The International Society for Optical Engineering

Estimating Speaker Scale Factors from Vowels

In the conventional maximum likelihood based speaker normalization approach, the optimal frequency warping factors are estimated by maximizing the likelihood of warped features in a grid search. The conventional method of likelihood computation for warped features does not account for the Jacobian of the transformation. This fact is pointed out by some researchers who have also shown that frequency warping is equivalent to the transformation in cepstral domain. As an approximation, variance normalization of cepstral features is used before likelihood computation to account for the Jacobian. In this paper, we suggest an alternate method to avoid the Jacobian problem. Our preliminary investigation shows that our proposed method provides improvement in normalization performance compared to the conventional method of warping factor estimation for a digit recognition task.