This is my experience as well. I was doing well in CS courses, but struggling in others, and didn't graduate early (well, didn't try to either). However, finishing whatever they threw at me allowed me to go ask for more, or try more harder things.
As a result, I learned way more than curriculum offered, got way harder assignments and projects (like designing a compression algorithm from ground up as a graduation project), and satisfied much more overall.
I continued my M.Sc. and Ph.D. at the same university (because I already had a job, and I just wanted to learn), and this head start (and being in good terms with the professors) allowed me to do similarly heavier curricula during these studies, too.
As a result, I learned way more than curriculum offered, got way harder assignments and projects (like designing a compression algorithm from ground up as a graduation project), and satisfied much more overall.
I continued my M.Sc. and Ph.D. at the same university (because I already had a job, and I just wanted to learn), and this head start (and being in good terms with the professors) allowed me to do similarly heavier curricula during these studies, too.
It's worth it.