Goodhart's law

Template:Short description Template:Distinguish Goodhart's law is an adage that has been stated as, "When a measure becomes a target, it ceases to be a good measure".<ref name="Strathern1997" /> It is named after British economist Charles Goodhart, who is credited with expressing the core idea of the adage in a 1975 article on monetary policy in the United Kingdom:<ref>Template:Cite encyclopedia</ref> Template:Blockquote

It was used to criticize the British Thatcher government for trying to conduct monetary policy on the basis of targets for broad and narrow money,<ref>Template:Cite book</ref> but the law reflects a much more general phenomenon.<ref>Template:Cite arXiv</ref>

Priority and background

File:Charles Goodhart delives the 2012 Long Finance conference keynote speech.JPG

Charles Goodhart, for whom the adage is named, delivering a speech in 2012

Numerous concepts are related to this idea, at least one of which predates Goodhart's statement.<ref>Template:Cite web</ref> Notably, Campbell's law likely has precedence, as Jeff Rodamar has argued, since various formulations date to 1969.<ref>Template:Cite journal</ref> Other academics had similar insights at the time. Jerome Ravetz's 1971 book Scientific Knowledge and Its Social Problems<ref name="SK">Template:Cite book</ref> also predates Goodhart, though it does not formulate the same law. He discusses how systems in general can be gamed, focuses on cases where the goals of a task are complex, sophisticated, or subtle. In such cases, the persons possessing the skills to execute the tasks properly seek their own goals to the detriment of the assigned tasks. When the goals are instantiated as metrics, this could be seen as equivalent to Goodhart and Campbell's claim.

Shortly after Goodhart's publication, others suggested closely related ideas, including the Lucas critique (1976). As applied in economics, the law is also implicit in the idea of rational expectations, a theory in economics that states that those who are aware of a system of rewards and punishments will optimize their actions within that system to achieve their desired results. For example, if an employee is rewarded by the number of cars sold each month, they will try to sell more cars, even at a loss.

While it originated in the context of market responses, the law has profound implications for the selection of high-level targets in organizations.<ref name="Goodhart1975" /> Jon Danielsson states the law as

And suggested a corollary for use in financial risk modelling:

Mario Biagioli related the concept to consequences of using citation impact measures to estimate the importance of scientific publications:<ref>Template:Cite journal</ref><ref>Template:Cite journal</ref>

Generalization

Later writers generalized Goodhart's point about monetary policy into a more general adage about measures and targets in accounting and evaluation systems. In a book chapter published in 1996, Keith Hoskin wrote:

In a 1997 paper on the misuse of accountability models in education, anthropologist Marilyn Strathern cited Hoskins expressing Goodhart's Law as "When a measure becomes a target, it ceases to be a good measure", and linked the sentiment to the history of accountability stretching back into Britain in the 1800s:

Examples

• The San Francisco Declaration on Research Assessment denounces several problems in science and as Goodhart's law explains, one of them is that measurement has become a target. The correlation between h-index and scientific awards is decreasing since widespread usage of h-index.<ref name='h-frac'>Template:Cite journal Companion webpage</ref>
• The International Union for Conservation of Nature's (IUCN) measure of extinction can be used to remove environmental protections, which resulted in IUCN becoming more conservative in labeling something as extinct.<ref>Template:Cite web</ref><ref>Template:Cite journal</ref>
• In healthcare, the misapplication of metrics can lead to adverse outcomes. For instance, hospitals striving to reduce length of stay (LOS) may inadvertently discharge patients prematurely, leading to increased emergency readmissions.<ref>Template:Cite web</ref>Template:Sps
• According to Tom and David Chivers in How to Read Numbers, the law applied to the British government response to the COVID-19 pandemic when it announced a target of 100,000 COVID-19 tests per day—initially a target for tests actually carried out and later for maximum capacity of test-taking. The number of useful diagnostic tests was far lower than the government-reported number when it announced it had met the target.<ref>Template:Cite book</ref>
• In the HBO television series The Wire, Roland 'Prezbo' Pryzbylewski draws parallels between public school policy of teaching to the test and improving crime statistics through reclassification of those crimes as "juking the stats". He notes that they both are examples of strategies that are ineffective at substantive improvements in learning and public safety respectively because they cater to ineffective metrics of standardized test scores and reported crime rates.<ref>Template:Cite web</ref>Template:Sps

See also

• Campbell's law – "The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures"
• Cobra effect – when incentives designed to solve a problem end up rewarding people for making it worse
• Confirmation bias – the tendency to search for and recall information that confirms or supports one's prior beliefs
• Gaming the system – manipulating rules and procedures to obtain a desired outcome
• Hawthorne effect – when people modify an aspect of their behavior in response to their awareness of being observed
• Lucas critique – the observation that it is naive to try to predict the effects of a change in economic policy entirely on the basis of relationships observed in historical data
• Map–territory relation – a type of reification fallacy where a model is confused with the thing being modeled
• McNamara fallacy – ignoring qualitative metrics on the basis that they cannot be measured
• Template:Annotated link
• Template:Annotated link
• Overfitting – an analysis that corresponds too closely or exactly to a particular set of data
• Peter principle – individuals are promoted based on success in their previous roles, and not the role of the new position
• Template:Annotated link
• Template:Annotated link
• Specification gaming – behaviour of artificial intelligence in working towards a poorly specified reward rather than the intended outcome
• Surrogation – in business, when a measure of a construct of interest evolves to replace that construct

Further reading

• Template:Cite book

External links

• Template:Cite web
• Template:Cite web
• Template:Cite podcast
• Template:Cite web History of thinking about Goodhart's Law