Couldn't it still be the case, though, that we have too much data, but it's also the wrong sort for actionable insights? As a scientist I find the most actionable data are often in the smallest, custom-made datasets, driven by some question, not trawling through masses of data collected without a goal, hoping that they'll have collected the right thing.

Sure, could very well be, and what is a perfect fit in one situation, might be unusable in another situation that at the surface looks like it's almost the same. It would be silly for me to claim that all data we need is already being collected or something like that. But that's not at odds with my abstract point that the realization that data is usually no longer the problem, but the lack of knowing what to do with it hasn't sunk in for most people. (This makes it sound like I think of myself as someone who has seen The Light and 'those others' are chumps, which would obviously be delusional of me, and I don't mean it that way)

I guess what I'm failing to articulate here is the shift that has snuck up on us over the last 10 or so years. My bitching about the quality of datasets today is about increasingly marginal issues (at the macro scale of course, there are still crap individual datasets, obviously); whereas 15 years ago, I didn't even have datasets to bitch about.

This has been my experience while taking masters courses, but I assumed it was because it's important to learn about all the available tools and techniques. Has there been no research on methodologies that aid in discovering insights?

It reminds me of the difficulties in teaching someone how to prove statements in math. The basic approach is to learn as many techniques as possible (tools in your tool bag), review existing proofs, and practice. You really can't teach someone how to find the connections and insights that lead to a proof. I was once told that was the art/creativity in proofs.