Interplay between timescales governs the residual activity of a harmonically bound active Brownian particle