This is not how I understood the argument of scientific stagnation though, wasn't it more to do with 'breakthrough' discoveries that somehow radically disrupted / changed our way of life?
In that regard I'm inclined to believe there is some sort of stagnation. Though not necessarily at the fault of the community or researchers. It could very well be a mix of what you said about problems being harder that we have, between 1900 - 2000 picked all the 'low hanging fruit' in physics, and that we now need lots of patient, 'baby step' type improvements to get to a new era where the technology is powerful enough to make big leaps again.
Kind of like how Deep Learning research stagnated due to lack of data volume and processing power.
This is my perspective, but I think it's totally up for debate and I'm keen to hear different opinions.
Between ~1880 and ~1920, the physical sciences made huge, crazypants leaps both in theory and experimentally. The rest of the 20th century was spent catching up with all the implications of those leaps. Now, the sciences have a set of very powerful ideas that have a bad habit of producing the right answers as far as anyone can see, but which have very visible holes and don't fit together. Thus, stagnation.
On the other hand, it may just be a return to the normal status quo.
Whether its the fault of the community or just natural consequences of the low hanging fruit being picked away will only be possible to know in retrospect, really. Is the issue that we don’t have the technology to figure things out, or that the community was too stuck in a local maximum of explainability to find the summit where everything ti explained? Until the next major breakthrough actually happens, it’s hard to know which is the case.
Although in general, I’d say it’s possible a big leap never happens again. Given that physics will ultimately be a finite set of rules, if we can explain the vast majority of phenomenon accurately, slotting the last few pieces into place might not grant us much. It will feel great for humanity to know of course, but it’s entirely plausible that the reason we have so much trouble figuring these these out is that they’re almost completely separate from the human experience. We might figure out quantum gravity, go “that’s nice”, but if it’s only relevant when there are stellar masses involved not be able to use it to change our way of life. Big changes to how humanity lives going forward could be entirely reliant on human invention/ingenuity, not us learning new facts about the universe we live in.
There are still poorly understood physical phenomena that could prove to be tremendously useful, like superconductors. If quantum gravity leads us to room-temperature superconductors (for example), that would be absolutely earth-shattering for humanity.
In that regard I'm inclined to believe there is some sort of stagnation. Though not necessarily at the fault of the community or researchers. It could very well be a mix of what you said about problems being harder that we have, between 1900 - 2000 picked all the 'low hanging fruit' in physics, and that we now need lots of patient, 'baby step' type improvements to get to a new era where the technology is powerful enough to make big leaps again.
Kind of like how Deep Learning research stagnated due to lack of data volume and processing power.
This is my perspective, but I think it's totally up for debate and I'm keen to hear different opinions.