Soft materials are known for their plethora of biomedical applications, intricate structure–property correlation and nonlinear mechanical response. Multiple length–time scale phenomena and hierarchical structure results in their nonlinearity. Phenomenological and continuum mechanical models have been developed to predict their mechanics, which have mostly been very material-specific with inability to predict the mechanics of different types of soft materials simultaneously. This shortcoming has been addressed in the present work, wherein a generic nonlinear viscoelastic model has been proposed to predict the mechanical response of hydrogels, sponges, and xerogels. A fractal derivative viscoelastic model is proposed considering a fractal Maxwell model in parallel with a nonlinear spring. In particular, this model is chosen to qualitatively mimic the material nonlinearity inherent in soft materials. The fractal dashpot in combination with the nonlinear spring accounts for the power law time-dependent rheology of generic soft materials. These two different aspects in the form of nonlinear stiffness and non-Newtonian rheology account for mechanics of most soft materials. The present model is shown to fit well the existing literature results for mechanical response of a multitude of soft material classes with different test conditions and loading rates, which is one of the salient features of the model, apart from its simplistic mathematical framework. Further, a parametric study is reported on the mechanics of nanocellulose loaded poly(vinyl alcohol) xerogel. The model predictions are observed to be in conjunction with the experimental observations. © 2021, The Author(s), under exclusive licence to Springer Nature B.V. part of Springer Nature.