Surely LLVM's inlining heuristics must be one of its strengths. I thought good inlining was almost all of optimisation these days, based on a Chandler Carruth talk on LLVM

If you are compiling a language like C++ and care a lot about code size, yes.

Most compilers don't have to, and then you just inline whatever you can until some estimated register pressure is reached at the call site.

Is MLIR practically good at anything other than loop patterns in AI and HPC acceleration?

It's more of a 'bring your own optimizer' kind of a framework.

The idea is that, you know best what optimizations work for your domain.

But a compiler needs a large amount of engineering for things which are not optimizations.

MLIR makes it possible to get this infra (developed utilizing lessons from LLVM and other compilers) for free and share improvements among multiple compilers without pulling your hair out trying to understand misleading academic papers.