The significance of operator training has dramatically increased due to complex automation strategies in modern process plants. It is reported that human errors account for 70% of the accidents in process industries, with inadequate training cited as one of the most common reasons for these incidents. Our previous work has shown the potential of eye-tracking to infer the mental state of control room operators. In this work, we propose a methodology that combines multi-scale data from the process simulator, control actions performed, and eye gaze data of the operators to evaluate their training outcomes. Specifically, we use fixation transition entropy, an eye-tracking metric, which can help infer the mental models of the process abnormalities developed by the operators during repeated control room tasks. Results indicate that the fixation transition entropy decreases on account of development of correct mental models of process while it remain at higher values when operator fails to update their mental models during plant abnormalities. Thus, the proposed metric can be used to gauge the development of operator's mental models during training to understand the transition from novice to becoming experts. © 2021 Elsevier B.V.