The rapid granular plane Couette flow is known to be unstable to pure spanwise perturbations (i.e.Â perturbations having variations only along the mean vorticity direction) below some critical density (volume fraction of particles), resulting in the banding of particles along the mean vorticity direction: this is dubbed 'vorticity banding' instability. The nonlinear state of this instability is analysed using quintic-order Landau equation that has been derived from the pertinent hydrodynamic equations of rapid granular fluid. We have found analytical solutions for related modal/harmonic equations of finite-size perturbations up to quintic order in perturbation amplitude, leading to an exact calculation of both first and second Landau coefficients. This helped to identify the bistable nature of nonlinear vorticity-banding instability for a range of densities spanning from moderately dense to dense flows. For perturbations with small spanwise wavenumbers, the bifurcation scenario for vorticity banding unfolds, with increasing density from the dilute limit, as supercritical pitchfork ˚ightarrow subcritical pitchfork ˚ightarrow subcritical Hopf bifurcations. The transition from supercritical to subcritical pitchfork bifurcations is found to occur via the appearance of a degenerate/bicritical point (at which both the linear growth rate and the first Landau coefficient are simultaneously zero) that divides the critical line into two parts: one representing the first-order and the other the second-order phase transitions. Both subcritical oscillatory and stationary solutions have also been uncovered for dilute and dense flows, respectively, when the spanwise wavenumber is large. In all cases, the nonlinear solutions correspond to inhomogeneous states of shear stress and pressure along the vorticity direction, and hence are analogues of vorticity banding in other complex fluids. The quartic-order mean-flow resonance is evidenced in the parameter space for which the second Landau coefficient undergoes a jump discontinuity of infinite order. The importance of retaining higher-order terms to calculate the second Landau coefficient and their possible effects on the nature of bifurcations are elucidated. © 2013 Cambridge University Press.