That's the core truth people don't seem to want to directly address. Most criticism of meritocracy approaches it from the angle that it is meant to give everyone an equal shot and has failed, when the real goal is to sift away those who can't contribute, at which it works marvelously (which isn't to say everyone who deserves a given shot gets one).
What some people seem to want instead is some sort of egalitocracy, but I have no idea how to make such a beast productive or useful, particularly for tech work, which has a fairly high intellectual bar.