The effect of initial grain size on microstructure and mechanical behaviour of Al-Mg (AA 5083) alloy was investigated by subjecting it to rolling at liquid nitrogen temperature (CR-cryorolling) up to various strains. Initial samples of different grain sizes, ∼ 12 μm and 100 μm and with 8 mm thickness were prepared by processing through rolling followed by annealing. Mechanical properties were investigated by employing hardness and tensile testing. Microstructural studies were carried out by using electron back-scattered diffraction (EBSD) and transmission electron microscopy (TEM) techniques. CR up to 2.3 true strain has resulted ultrafine grained (UFG) structure with improved mechanical properties. Effect of annealing on microstructure and mechanical properties of the material with different initial grain sizes CRed up to true strain 2.3 was investigated by performing short annealing at 300 °C for 5 min. At lower strain (true strain-0.3) the change in hardness and ductility was not significant due to low dislocation density. On further increase in strain (true strain-2.3), the fine grained material has shown improved hardness (134 HV) and ultimate tensile strength (UTS) (510 MPa), as compared to coarse grain material with hardness (125 HV) and UTS (475 MPa). It has been observed that during annealing the microstructure evolution through recovery and recrystallization progresses faster in fine grained material due to large volume fraction of grain boundaries which acts as active sites for nucleation. Fully homogenous ultrafine grained structure with grain size in the range of 100-500 nm was obtained in fine grained material. The role of second phase particles on grain refinement was also investigated.