Also Fugaku (The world's #1 supercomputer from 2020 till present) uses HBM2 for its CPU <-> Memory.

https://en.wikipedia.org/wiki/Fugaku_(supercomputer)

(On-package, rather than on-die.)

Wow, okay then. I wonder if that sort of setup will be like the quad itanium motherboard setups that needed massive sustained airflow to keep running...

Just like any respectable CPU they will implement thermal throthling.