In this paper, we propose an approach to acoustic modeling using vector quantization in a Mercer kernel feature space to obtain a sequence of codebook indices, and then use a support vector machine based classifier to classify the sequence of codebook indices. Clustering and vector quantization in the kernel feature space induced by a nonlinear innerproduct kernel is helpful in proper separation of nonlinearly separable clusters in the input acoustic feature space. Effectiveness of the proposed approach to acoustic modeling is demonstrated for recognition of spoken letters in E-set of English alphabet, and for recognition of a large number of consonant-vowel type subword units in continuous speech of three Indian languages. Performance of the proposed approach to acoustic modeling is compared with that of a continuous density hidden Markov model based classifier in the input acoustic feature space. Though there is a significant loss of information due to discretization involved in vector quantization, the proposed approach gives a performance better than that of classifiers using the continuous valued acoustic feature representation. ©2007 IEEE.