My issue with both this and u-substitution is that you don't know what expression to use. There are a LOT of expressions that plausibly simplify the integral. But you have to do a bunch of algebra for each one (and not screw it up!), without really knowing whether it actually helps.
OTOH, if I'm given the expression, it's just mechanical and unrewarding.
I see your point, but as it is stated in the article, it is one of those techniques that require practice, and time to mature. And like it mentions, it's a bit like chess...when you're presented with some troubling integral, you can parametrize it in a number of ways. Most will bring you back to the beginning (like with the standard integration by parts), but the right one will make your life much easier.
It can be frustrating when math does not have any clear single path, but that's just the nature of the beast. In the beginning you'll just have to explore all the paths, but do that a couple of hundred times, and you'll start to notice patterns and what will work / what will not. Kind of like chess, where a good chess player can think N moves ahead in time.
I don't know about this particular case though, I get the feeling there's a system to it that can be exploited by eg Wolfram. It's just that you're in the dark for a long time before you find the switch.
Your intuition is right. There is a general algorithm for finding the antiderivatives: https://en.wikipedia.org/wiki/Risch_algorithm Its simplified form can solve pretty much all the undergrad antiderivation problems.
I'm a math major, but I consider the time spent learning the tricks for antiderivation to be kinda useless.
It doesn't. But if there is an elementary antiderivative, the Risch algorithm will find it (given the caveats listed in the Wikipedia article). But it might require a lot of substitutions, making its manual application impractical.
Another caveat is that Risch algorithm applies only to antiderivatives, not to the definite integrals. Some definite integrals can be computed without finding the antiderivative, often with the help of Feynman's trick.
“Perhaps I could best describe my experience of doing mathematics in terms of entering a dark mansion. One goes into the first room, and it’s dark, completely dark. One stumbles around bumping into the furniture, and gradually, you learn where each piece of furniture is, and finally, after six months or so, you find the light switch. You turn it on, and suddenly, it’s all illuminated. You can see exactly where you were.” - Andrew Wiles
OTOH, if I'm given the expression, it's just mechanical and unrewarding.