A working aero engine rotor system is subjected to multi-objective optimisation using Genetic Algorithm based optimisation. A Hybrid Genetic Algorithm (HGA) is introduced to reduce the weight and unbalance response of the rotor system with constrain on critical speed. The existing aero engine gear box casing vibration is found to be within the critical speed constraint and additional constraints are imposed to move the critical away from this zone. Bearing-pedestal model and Rayleigh damping model are used for accurate results. The optimisation resulted in Pareto optimal solutions and best solution selected using utopia point concept. The outcome of the paper is a comparative study which highlights the advantages of HGA over Controlled Elitist Genetic Algorithm (CEGA) and Goal Programming (GP). © 2019 IOP Publishing Ltd. All rights reserved.