The aim of this paper is to construct the parabolic version of the Donaldson-Uhlenbeck compactification for the moduli space of parabolic stable bundles on an algebraic surface with parabolic structures along a divisor with normal crossing singularities. We prove the non-emptiness of the moduli space of parabolic stable bundles of rank 2.