Dynamic kernel (DK)-based support vector machines are used for the classification of varying length patterns. This paper explores the use of intermediate matching kernel (IMK) as a DK for classification of varying length patterns of long duration speech represented as sets of feature vectors. The main issue in construction of IMK is the choice for the set of virtual feature vectors used to select the local feature vectors for matching. This paper proposes to use components of class-independent Gaussian mixture model (CIGMM) as a representation for the set of virtual feature vectors. For every component of CIGMM, a local feature vector each from the two sets of local feature vectors that has the highest probability of belonging to that component is selected and a base kernel is computed between the selected local feature vectors. The IMK is computed as the sum of all the base kernels corresponding to different components of CIGMM. It is proposed to use the responsibility term weighted base kernels in computation of IMK to improve its discrimination ability. This paper also proposes the posterior probability weighted DKs (including the proposed IMKs) to improve their classification performance and reduce the number of support vectors. The performance of the support vector machine (SVM)-based classifiers using the proposed IMKs is studied for speech emotion recognition and speaker identification tasks and compared with that of the SVM-based classifiers using the state-of-the-art DKs. © 2012 IEEE.