AIO performance on CPUs is largely limited by thermal transfer through the coldplate/IHS (Integrated Heat Spreader), not by radiator size. Basically the radiator is keeping the fluid very cool already, but heat can't move through the IHS quickly enough. So it takes a large improvement in fluid temperature to make a small improvement in die temperature - you are "pushing on a string" as the expression goes.
Almost no AIOs have the fluid-temperature sensors that would allow you to measure this directly, so everyone uses the die sensors. Which, since they're behind the IHS, will be much higher than the fluid itself. The die temperature is also a measurement of interest too - I'm merely explaining why the number you're seeing in the die sensor isn't really the big picture of how good the the radiator is doing at cooling. The die is hot, but the fluid is cool.
The AMD 295x2 is an extremely good example of this observation - this card had a single 120mm radiator, with one fan, and it could dissipate >500W of heat at ~60C die temperature. (not sure if this source says this directly but the non-OC power was ~430W average during gaming, and ~250W is not unreasonable for each 290X chip - actually they could go to 300W or higher if you really poured it on, but they also did generally show some significant power scaling with temperatures, so ~250W per chip/500W total is a reasonable estimate imo).
You might say - but that's a dual-GPU card, with bare dies. And yes, that's my point, when the coldplate/IHS is no longer a bottleneck moving heat into the loop, a 120mm radiator is comfortably capable of dissipating 500W of power back out of the loop at extremely reasonable operating temperatures (60C die temperature). 60C is actually barely breaking a sweat, you could probably do 1000W through that 120mm if you didn't mind a die temperature in the 80-90C range. In CPU overclocking - your power limits/temps are almost entirely limited by how fast you can get that heat through the IHS. Reducing fluid temps (by increasing radiator size) is pushing on a string, it takes big gains in fluid temp to produce a small improvement in die temp.
Incidentally, direct-die cooling is the last untapped frontier of gains for ambient (non-chilled) overclocking. Der8auer and IceManCooler.com both make "support brackets" that replace part of the ILM (integrated loading mechanism - the socket and its tensioning mechanism and attachment to the motherboard) that holds the processor. This is necessary since the IHS is actually part of the ILM - the ILM presses down on the sides of the IHS, so removing it would change the pressure, and the ILM needs to keep a specific level of pressure on the chip to make a good contact with the pins but but without damaging anything. But you can delid the processor (there are services that do this for soldered chips, I don't recommend doing it at home) and use one of those brackets with a "normal" waterblock/AIO (or even air-cooler), since the bracket is holding the chip in the pin-bed at the proper tension.
Thermal density is going nowhere but up, Dennard scaling is over, so that is the only way to really improve thermals on <= 7nm-class nodes. Even AMD runs hot - they routinely run in the mid-80C range nowadays, even though they don't pull a lot of power - because of that thermal density, and every time they shrink it's going to get worse. The gains may be more worth it on Intel though - they show better scaling from power/voltage, TSMC nodes seem to pretty much top out at about 4 GHz and past there it gets exponentially worse for very little actual performance gain. 4.3, 4.4, sure, but they don't seem to do 5-5.3 GHz like Intel can on their Intel 7 given good temps and enough voltage.
But yes, to go back to your original point, I really like my 3090 Kingpin as well. It runs extremely cool, I can keep the die at literally 30C with the fans cranked all the way up, and it'll keep the VRAM at under 70C (!). And since it is a 2-slot card it doesn't turn into a compatibility mess with motherboard pcie slots getting blocked and needing airspace/etc. I am 100% behind AIOs on the larger gpus that we are seeing lately, this is a better solution than triple-slot or 3.5 slot coolers, which are (imo) completely ridiculous.
Then you're not paying the Kingpin markup if you're not planning to x-oc it. I did that on a Titan X (Pascal) which was still using a blower design and it worked fantastically.
This is true, but: you have to watch compatibility (note that there are no 3000 series chips on that list - because NVIDIA changed their hole placement again, and AMD has a couple different sizes for their different chips, 6500XT/6400 is definitely smaller for example), and also it doesn't do as good a job cooling VRAM. You can put add-on heatsinks on the VRAM chips, but they can fall off and short something. And adding them on the back can run into compatibility problems with bumping into the CPU heatsink.
And VRAM temperatures are a big problem on the Ampere cards - I don't really think running >100C all the time is really gonna be great for them long-term. Even gaming (vs mining) it's not abnormal to see VRAM over 100C (especially 3090, with the chips on the back, but also on the other GDDR6X cards, GDDR6X just runs extraordinarily hot). I know what NVIDIA and Micron say, I'm not sure I believe it. Above-100C is really really dubious imo.
for the 3090, with the VRAM on the back, I think it makes sense to go with a factory-configured AIO. Other cards, and especially GDDR6 cards, sure, it does work and it does help. Don't go too nuts tightening the AIO down though (ask me why! >.<)
Gelid used to make nice little cooling shields for the VRM and memory modules. I'm disappointed they stopped, although I'm sure it was a tiny market. For single-sided cards that is a much much nicer solution than stick-on heatsinks imo.
When I ran it the vram temps were actually fine (although not the scorching GDDR6X variety of course). Since it's not sharing thermal mass with the GPU die, the direct airflow on its own was plenty.
You actually can see this with the 3090 in fact. Simply pointing a fan at the back of the card does wonders and easily keeps them in spec without a heatsink at all, although the backplate is acting as a bit of a spreader. Which makes sense since each memory chip is only like 2-3W. You don't need a heatsink for that, just a little bit of airflow
Thanks for the detailed write up, I learnt something new today.
I sometimes feel I should stop wasting so much time on the internet, but sometimes I realize that it means missing out on such comments that distill a lot of/important information.
Almost no AIOs have the fluid-temperature sensors that would allow you to measure this directly, so everyone uses the die sensors. Which, since they're behind the IHS, will be much higher than the fluid itself. The die temperature is also a measurement of interest too - I'm merely explaining why the number you're seeing in the die sensor isn't really the big picture of how good the the radiator is doing at cooling. The die is hot, but the fluid is cool.
The AMD 295x2 is an extremely good example of this observation - this card had a single 120mm radiator, with one fan, and it could dissipate >500W of heat at ~60C die temperature. (not sure if this source says this directly but the non-OC power was ~430W average during gaming, and ~250W is not unreasonable for each 290X chip - actually they could go to 300W or higher if you really poured it on, but they also did generally show some significant power scaling with temperatures, so ~250W per chip/500W total is a reasonable estimate imo).
https://www.techpowerup.com/review/amd-r9-295-x2/28.html
You might say - but that's a dual-GPU card, with bare dies. And yes, that's my point, when the coldplate/IHS is no longer a bottleneck moving heat into the loop, a 120mm radiator is comfortably capable of dissipating 500W of power back out of the loop at extremely reasonable operating temperatures (60C die temperature). 60C is actually barely breaking a sweat, you could probably do 1000W through that 120mm if you didn't mind a die temperature in the 80-90C range. In CPU overclocking - your power limits/temps are almost entirely limited by how fast you can get that heat through the IHS. Reducing fluid temps (by increasing radiator size) is pushing on a string, it takes big gains in fluid temp to produce a small improvement in die temp.
Incidentally, direct-die cooling is the last untapped frontier of gains for ambient (non-chilled) overclocking. Der8auer and IceManCooler.com both make "support brackets" that replace part of the ILM (integrated loading mechanism - the socket and its tensioning mechanism and attachment to the motherboard) that holds the processor. This is necessary since the IHS is actually part of the ILM - the ILM presses down on the sides of the IHS, so removing it would change the pressure, and the ILM needs to keep a specific level of pressure on the chip to make a good contact with the pins but but without damaging anything. But you can delid the processor (there are services that do this for soldered chips, I don't recommend doing it at home) and use one of those brackets with a "normal" waterblock/AIO (or even air-cooler), since the bracket is holding the chip in the pin-bed at the proper tension.
Thermal density is going nowhere but up, Dennard scaling is over, so that is the only way to really improve thermals on <= 7nm-class nodes. Even AMD runs hot - they routinely run in the mid-80C range nowadays, even though they don't pull a lot of power - because of that thermal density, and every time they shrink it's going to get worse. The gains may be more worth it on Intel though - they show better scaling from power/voltage, TSMC nodes seem to pretty much top out at about 4 GHz and past there it gets exponentially worse for very little actual performance gain. 4.3, 4.4, sure, but they don't seem to do 5-5.3 GHz like Intel can on their Intel 7 given good temps and enough voltage.
But yes, to go back to your original point, I really like my 3090 Kingpin as well. It runs extremely cool, I can keep the die at literally 30C with the fans cranked all the way up, and it'll keep the VRAM at under 70C (!). And since it is a 2-slot card it doesn't turn into a compatibility mess with motherboard pcie slots getting blocked and needing airspace/etc. I am 100% behind AIOs on the larger gpus that we are seeing lately, this is a better solution than triple-slot or 3.5 slot coolers, which are (imo) completely ridiculous.