This paper presents use of a cost graph as a representation of a multi-floor building to enable the multi-floor autonomous navigation capability for a team of robot(s). A method for global path planning on this cost graph have been presented. A navigation stack provides a framework for building autonomous navigation capabilities. A navigation stack which enables use of the proposed approach for navigation in a multi-floor building and enables multi-robot operations has been detailed. The improvements provided by the proposed navigation stack over the existing ROS (Robot Operating System) navigation stack have been explained. A way to integrate multiple local path-execution nodes which can combine together to execute the planned global path has also been explained. The paper also demonstrates the reuse of existing ROS compliant source codes for implementation of the proposed navigation stack, thereby optimizing the use of proven and established technology. Further, the extensions to different components of the existing ROS navigation stack, definition of new ROS messages and action definitions, to enable interaction between the components of the stack has been explained. The paper concludes with a brief study on how the proposed stack can be used for multi-robot operations.