Periodic motions of the nonlinear system representing the escape equation with cosine and sine parametric excitations and external harmonic excitations are obtained by the incremental harmonic balance (IHB) method. The system contains quadratic stiffness terms. The Jacobian matrix and the residue vector for the type of nonlinearity with parametric excitation are explicitly derived. An arc length path following procedure is used in combination with Floquet theory to trace the response diagram and to investigate the stability of the periodic solutions. The system undergoes chaotic motion for increase in the amplitude of the harmonic excitation which is investigated by numerical integration and represented in terms of phase planes, Poincare sections and Lyapunov exponents. The interpolated cell mapping (ICM) method is used to obtain the initial condition map corresponding to two coexisting period 1 motions. The periodic motions and bifurcation points obtained by the IHB method compare very well with results of numerical integration.