In this paper we propose an adaptive training method for parameter estimation of acoustic models in the speech recognition system. Our technique is inspired from the Cluster Adaptive Training (CAT) method which is used for rapid speaker adaptation. Instead of adapting the model to a speaker as in CAT, we adapt the parameters of the context dependent triphone states (tied states) from context independent states (monophones). This is achieved by finding a global mapping of parameters of the tied state from the parametric subspace of monophone models. This technique is similar to Subspace Gaussian Mixture Model (SGMM), but differs in the initialization of parameters and in the update of weights of Gaussian mixture components. We show that, the proposed method can match the performance of the conventional HMM system for large amount of training data and outperforms it when the number of training examples are less. © 2013 IEEE.