Having greater than 50% of the computing power of the bitcoin network does not give you complete power of the currency. You would be able to prevent transactions from occurring, and double-spend recent transactions. Since the bitcoin network is effectively the most powerful supercomputer in the world, this would be a very expensive attack to sustain, without that large of a payoff.
I definitely wouldn't recommend starting with the official bitcoin paper to learn about this. That would be like telling a math student to just read Newton if they want to learn Calculus :). Check out the bitcoin wiki instead: https://en.bitcoin.it/wiki/Weaknesses#Attacker_has_a_lot_of_...
FWIW, when I was figuring out Bitcoin I avoided the paper at first (due to logic similar to yours) but eventually read it because there was a lot of noise in the wiki, etc (may have changed). I found it surprisingly accessible; more so than the vast majority of academic papers I've come across (I have most of an EE degree but little to no formal CS training).
I definitely wouldn't recommend starting with the official bitcoin paper to learn about this. That would be like telling a math student to just read Newton if they want to learn Calculus :). Check out the bitcoin wiki instead: https://en.bitcoin.it/wiki/Weaknesses#Attacker_has_a_lot_of_...