In this letter, we present passivity-based convergence analysis of continuous time primal-dual gradient method for convex optimization problems. We first show that a convex optimization problem with only affine equality constraints admits a Brayton Moser formulation. This observation leads to a new passivity property derived from a Krasovskii-type storage function. Second, the inequality constraints are modeled as a state dependent switching system. Using tools from hybrid systems theory, it is shown that each switching mode is passive and the passivity of the system is preserved under arbitrary switching. Finally, the two systems: 1) one derived from the Brayton Moser formulation and 2) the state dependent switching system, are interconnected in a power conserving way. The resulting trajectories of the overall system are shown to converge asymptotically, to the optimal solution of the convex optimization problem. The proposed methodology is applied to an energy management problem in buildings and simulations are provided for corroboration. © 2017 IEEE.