With the emergence of IoT, the number of devices that is getting connected is increasing exponentially, which poses a constraint on the lower latency requirements of processing these tasks. Evolving technologies such as fog computing and edge computing consist of computationally lesser exhaustive power servers, which bring this processing near the edge or onto the devices, thereby reducing the round-trip delay as well as the load on the entire network. In this work, we propose the hierarchical arrangement of servers in the fog computing layer for this processing of data and derive the analytical framework for the same. The proposed architecture works on top-down scheduling policy rule for the incoming packets, defined mathematically in terms of two-dimensional Markov chains. The performance of the proposed architecture is compared with an equivalent flat architecture analytically in terms of the mean sojourn time and the mean computational power and justified using simulation results.