The Tyranny of Metrics

I'm all for metrics, so having some arguments on what to look for to avoid their pitfalls is useful.
I think it is a useful book and correctly points to when metrics have been used wrongly, specially in the public realm.

Some sources of dysfunction

If promotion/salary relies heavily on metrics then people may distort their work incurring in different kind of schemes.
- Do work that affects the metric only, and stop working on more important but not measured tasks.
- Work on easy targets only to improve some ratio. For instance a school that does not accept problematic kids to get better graduation rates.
- Lower standards. If success ratio < 2s is too low, just change to 2.5seconds and you improved your SLO. Airlines adding 30min to flytime just so they can land ahead of time every time.
- Change classification of tasks. For instance police booking felonies as misdemeanors to “improve” crime rate in the area.
- If a lot of money is on the line, some people will simply doctor the numbers

“Some metrics are better than no metrics” is commonly said what in practice means there is no dedicated time or resources, so only what's easily measurable gets measured. Instead of what's important to measure. Again focus falls into what's not important.

Related: Many roles at work has a lot of types of tasks. It is very difficult to measure 360 degrees of that, measuring only partially distorts how people perform those roles. Example: measure code commits for engineers.

Picking metrics that work somewhere else and applying them in your context. Because they are coming from the industry they look authoritative but they could totally miss the mark in your particular environment and be counterproductive.

Solution

The book does not give much in terms of solutions but the one thing it puts forward is:
Create metrics that are created in the consensus of the experts, bottom-up , not top down.
So for instance each engineering team would gather and agree on what makes sense for them to track and hold accountable for, instead of the CTO or something saying every engineering team will be measured by this or that.