I'm the same way. I love playing sports personally. I'm more into individual sports like racket, cycling, etc, though I have played soccer and such way back. But I just can't really get into spectating and never had. For me I'd much rather be doing my own thing than watching somebody else do theirs; I lose interest pretty quickly. That includes both e-sports and physical sports. I'll enjoy the odd game day BBQ and beers with the guys though.
I can't get into either though. I think it has more to do than intellectuals not liking spectating sports.