This work deals with processing of nanocomposite gel coat system using unsaturated polyester resin, aerosil powder and dodecylamine treated nanoclays. Further, the work focuses on the influence of nanoclay addition on mechanical and water barrier properties of a polyester gel coat system with different weight fractions (1, 2, 3, 4 & 5 wt.%) of nanoclay. The nanocomposite gel coat system exhibits 94% improvement in tensile modulus for 5wt.% nanoclay and 24% improvement in tensile strength for 2wt.% nanoclay. Nanocomposite gel coat system exhibit a maximum of 68% improvement in barrier resistance for 3 wt.% nanoclay compared to the conventional gel coat system. At higher clay concentration (>3%) the mechanical and barrier properties decrease due to the limitation in the processing as well as the high viscosity of resin/clay/aerosol system.