The primordial scalar power spectrum is well constrained by the cosmological data on large scales, primarily from the observations of the anisotropies in the cosmic microwave background. Over the last few years, it has been recognized that a sharp rise in power on small scales will lead to the enhanced formation of primordial black holes (PBHs) and also generate secondary gravitational waves (GWs) of higher and, possibly, detectable amplitudes. It is well understood that scalar power spectra with COBE normalized amplitude on the cosmic microwave background scales and enhanced amplitudes on smaller scales can be generated due to deviations from slow roll in single, canonical scalar field models of inflation. In fact, an epoch of so-called ultraslow roll inflation can lead to the desired amplification. We find that scenarios that lead to ultraslow roll can be broadly classified into two types, one wherein there is a brief departure from inflation (a scenario referred to as punctuated inflation) and another wherein such a departure does not arise. In this work, we consider a set of single field inflationary models involving the canonical scalar field that lead to ultraslow roll and punctuated inflation and examine the formation of PBHs as well as the generation of secondary GWs in these models. Apart from considering specific models, we reconstruct potentials from certain functional choices of the first slow roll parameter leading to ultraslow roll and punctuated inflation and investigate their observational signatures. In addition to the secondary tensor power spectrum, we calculate the secondary tensor bispectrum in the equilateral limit in these scenarios. Moreover, we calculate the inflationary scalar bispectrum that arises in all the cases and discuss the imprints of the scalar non-Gaussianities on the extent of PBHs formed and the amplitude of the secondary GWs generated. We conclude with a discussion on the wider implications of our results. © 2021 American Physical Society.