Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

This is a nice refinement to the discussion. There is an element of truth to Goodhart's law, but it is certainly not a law and it is maybe not even a default condition. As per the author, the work of Demming clearly shows that measuring tolerances has a good effect on quality. I personally even seen Goodhart's law (mis)applied to discussions of machine learning pipelines, so clearly there are glaring exceptions to the utility of metrics.


> Goodhart's law . . . is certainly not a law

A scientific law is a description of a common pattern. It's not a rule voted into being by the reality legislature and enforced by nature beat cops.

Goodhart's certainly is a law.

> Goodhart's law (mis)applied to discussions of machine learning pipelines, so clearly there are glaring exceptions to the utility of metrics.

How can an alleged misapplication of Goodhart's law mean that metrics are not always useful? Goodhart's law itself says metrics aren't always useful.


It's not a scientific law. It's a meme law, broadly applicable but not always true.

Scientific laws have no counterexamples, only refinements.


Goodhart was an economist who stated his eponymous law in 1975, before Dawkins's The Selfish Gene (origin of "meme") was published in 1976. So Goodhart was not meme-ing.

If you want to argue that economics isn't science or isn't hard science, fine.

But law is used here in the sense of scientific law nevertheless.

https://en.wikipedia.org/wiki/Category:Economics_laws


Yes, for Deming:

Enumerative study: A statistical study in which action will be taken on the material in the frame being studied.

Analytic study: A statistical study in which action will be taken on the process or cause-system that produced the frame being studied. The aim being to improve practice in the future.

https://en.wikipedia.org/wiki/Analytic_and_enumerative_stati...


> Goodhart's law (mis)applied to discussions of machine learning pipelines

I see a way in which it overlaps with this issue https://en.wikipedia.org/wiki/Overfitting


For context this was just a team objecting to tracking progress against an external standard. The proposed metric was not that far off from the mean square error loss that was used for training models. Goodhart's law was just used as a rhetorical device to evade accountability.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: