An analytical framework is developed to understand and predict the thermoacoustic instability in solid rocket motors, taking into account the non-orthogonality of the eigenmodes of the unsteady coupled system. The coupled system comprises the dynamics of the acoustic field and the propellant burn rate. In general, thermoacoustic systems are non-normal leading to non-orthogonality of the eigenmodes. For such systems, the classical linear stability predicted from the eigenvalue analysis is valid in the asymptotic (large time) limit. However, the short-term dynamics can be completely different and a generalized stability theory is needed to predict the linear stability for all times. Non-normal systems show an initial transient growth for suitable initial perturbations even when the system is stable according to the classical linear stability theory. The terms contributing to the non-normality in the acoustic field and unsteady burn rate equations are identified. These terms, which were neglected in the earlier analyses, are incorporated in this analysis. Furthermore, the short-term dynamics are analysed using a system of differential equations that couples the acoustic field and the burn rate, rather than using ad hoc response functions which were used in earlier analyses. In this paper, a solid rocket motor with homogeneous propellant grain has been analysed. Modelling the evolution of the unsteady burn rate using a differential equation increases the degrees of freedom of the thermoacoustic system. Hence, a new generalized disturbance energy is defined which measures the growth and decay of the oscillations. This disturbance energy includes both acoustic energy and unsteady energy in the propellant and is used to quantify the transient growth in the system. Nonlinearities in the system are incorporated by including second-order acoustics and a physics-based nonlinear unsteady burn rate model. Nonlinear instabilities are analysed with special attention given to pulsed instability. Pulsed instability is shown to occur with pressure coupling for burn rate response. Transient growth is shown to play an important role in pulsed instability. © 2010 Cambridge University Press.