In this study, optimization of planetary gearbox considering regular mechanical and critical tribological constraints such as wear and scuffing is done. The design variables considered are the number of teeth in the sun, planet, ring gear, module, face width, diameter of shafts, planet pin diameter, the hardness of gear material and kinematic viscosity of the oil. The two objective functions are the weight and total power loss of the planetary gearbox. The analysis is done for the unmodified gear tooth profile with various input power from 30 to 90 kW with an incremental of 30 kW at the input speed of 1000 rpm. The result of multi-objective optimization with tribological constraints is compared with the single objective optimization considering the weight of the gearbox as the objective with and without tribological constraints. The result obtained considering multi-objective optimization shows that there is a significant reduction in weight and power loss as compared to results obtained by considering single objective optimization with and without tribological constraints. Further, the risk of gear failure in wear is in moderate to high range when the tribological constraints are not considered. © 2021, The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.