And yet when I got bored during the COVID lockdown and decided to analyse the published data sets against infection spreading models such as SIR, to my horror I discovered that every published data set had something Stupid about it with a capital S. Most commonly it was transposed data, published with each day's data in columns instead of rows.
I remember one official announcement from a state government health department that was investing significant money into developing a "scalable solution" because... they hit the 16K Excel maximum column count. Of course, they could have simply put their data into rows and "scaled" their existing solution to 1M data points, but they'd much rather pay Deloitte, Accenture, or whomever a couple of million dollars for a real enterprise system instead.
Next time I come across idiocy like this, I'm going refer back to this article and point to the four thousand year old tablet and say: "Those people got it! They understood how to do this! Why haven't you caught up to technology that was around before widespread adoption of the wheel!?"
The problem is mostly that some structure that looks good at start, looks bad after a while of using.
Maybe the first data was on postit notes. As the pandamic kept returning in waves, they thought they could use data in excel with new dates per row. Then new beta, delta,... variants emerged and they ran out of horizontal screen real estate.
I remember one official announcement from a state government health department that was investing significant money into developing a "scalable solution" because... they hit the 16K Excel maximum column count. Of course, they could have simply put their data into rows and "scaled" their existing solution to 1M data points, but they'd much rather pay Deloitte, Accenture, or whomever a couple of million dollars for a real enterprise system instead.
Next time I come across idiocy like this, I'm going refer back to this article and point to the four thousand year old tablet and say: "Those people got it! They understood how to do this! Why haven't you caught up to technology that was around before widespread adoption of the wheel!?"