In order to obviate the requirement of drift dynamics in adaptive dynamic programming, integral reinforcement learning (IRL) has been proposed as an alternate formulation of Bellman equation. However control coupling dynamics is still needed to obtain closed-form expression of optimal control effort. In addition to this, initial stabilizing controller and two sets of neural networks (NN) (known as Actor-Critic) are required to implement IRL scheme. In order to remedy these, this paper presents a critic-only IRL controller coupled with an experience replay (ER)-based identifier to solve optimal tracking control problem for an unknown continuous-time systems under actuator constraints. The presented control architecture is shown to yield tighter residual sets for state tracking error and error in NN weights. The simulation results establish the efficacy of the presented control scheme on continuous-time systems. © 2021 Informa UK Limited, trading as Taylor & Francis Group.