You'll never know 'til you try? Sure, but most people get plenty of chance to try mathematics at school. And while school-level mathematics ain't the same as research-level mathematics, enough of it uses the same bits of brain that by the time you hit university you already know whether you're brilliant at mathematics or not.
Terry Tao was clearly brilliant at mathematics as a child, though what nobody knew then was that he'd go on to be super-brilliant at mathematics as an adult, rather than being one of the many disappointing prodigies.
Anyway, I'm pretty smart, but I don't take career advice from anyone as smart as Terry Tao, just as I don't take dating advice from Scarlett Johannsen.
>And while school-level mathematics ain't the same as research-level mathematics
School mathematics isn't even math, it's a bad joke. I recommend Lockhart's Lament[1],
I fully share his views. School 'math' destroyed every ounce of interest I had in math,
and now, as a CS student, I still have a hard time getting rid of this attitude towards
the subject even thought I know a lot better now.
If you enjoy it, do it.