Temperature changes are exhibited by a material when subjected to mechanical loads in the elastic as well as the plastic regimes. In this paper, we analyze the observed thermo-mechanical phenomenon from elastic cyclic loading tests (stress below yield point) conducted on stainless steel (SS304) using two well-known rheological models viz., Kelvin-Voigt model and Maxwell's models. The Kelvin-Voigt model is shown to be well-suited in characterizing the mechanical as well as the associated thermal response. In seeking a deeper basis for the success of the Kelvin-Voigt model, correlations are sought between the model's key parameter - viscosity and the material's microscopic property viz. the grain boundary sliding coefficient. A plausible description is offered for the ability of Kelvin-Voigt model to explain the thermo-mechanical response under elastic cyclic loading. The effect of grain size on thermomechanical response and the variation of grain boundary diffusion coefficient with applied load is demonstrated, theoretically. The new description is used to predict the thermo-mechanical behavior of various other polycrystalline materials such as aluminum. Based on the models developed, experiments are proposed for further research. © 2012 Elsevier B.V.