A smart city aims at providing effective services to its citizens and administrators through the Internet of things (IoT) paradigm. A typical smart city architecture consists of a large topology with heterogeneous data generating sensors and communication technologies with support for multiple applications such as traffic management, smart energy, smart homes, and industrial IoT. Each application has a unique data generation rate and latency constraint based on the criticality of the application. In traditional IoT architecture, the data is processed in the Cloud which results in high propagation delay and hence high latency. To alleviate this problem, a decentralized and distributed paradigm called Fog Computing (FC) is gaining importance in latency critical IoT networks. The Fog layer is an intelligent layer in which the intermediate gateways are smart and have processing and storage resources. Smart processing of data at the fog layer results in significant reduction in latency. However, the smartness of gateways comes with a price. Therefore, placement of these resourceful gateways has to be done in a cost efficient manner considering latency constraints of all the applications supported in the system. In this paper, we propose a Hungarian method based Topology Construction (HTC) algorithm to identify the optimal locations and resources for the gateways such that over all system cost is minimized and latency constraints are met. We perform exhaustive simulations to show the effectiveness of our algorithm by comparing with existing models. © 2018 IEEE.