Tyranny of Metrics

EconTalk recently had Jerry Muller on the podcast discussing his book "Tyranny of Metrics". He laments over our fixation with our obsession with measurement.

Metrics are fine as a diagnostic tool, he argues, but when they are used as surrogates for success and attached to rewards, things go haywire. There are only a handful of metrics that retain their usefulness once they are widely adopted. The power of incentives prods people to game the statistic.

Muller gives several examples from various fields:

Measuring the success rates of surgeons performing certain operations and making them available seems like a good idea. Undoubtedly, transparency would helps patients make better choices. However, after the scorecards became public, surgeons began avoiding complicated cases, which would lower their batting average.

Then there is the phenomenon of goal diversion. A great deal of K-12 education has been distorted by the emphasis that teachers are forced to place on preparing students for standardized tests of English and math, where the results of the tests influence teacher retention or school closings. Teachers are instructed to focus class time on the elements of the subject that are tested (such as reading short prose passages), while ignoring those elements that are not (such as novels). Subjects that are not tested—including civics, art, and history—receive little attention. 

Or, to take an example from the world of business. In 2011 the Wells Fargo bank set high quotas for its employees to sign up customers who were interested in one of its products (say, a deposit account) for additional services, such as overdraft coverage or credit cards. For the bank’s employees, failure to reach the quota meant working additional hours without pay and the threat of termination. The result: to reach their quotas, thousands of bankers resorted to low-level fraud, with disastrous effects for the bank. It was forced to pay a fortune in fines, and its stock price dropped.

Here is yet another book review in Science.