People love measuring things. We’re constantly looking for ways to make analysis less subjective, and apply hard data to decision making. Too much measuring, though, leads to oversimplification.
Ask an economist and they‘ll tell you how unpredictable large economies are. Programmers will tell you that it’s relatively easy to make computer programs until you account for the humans who use them, because even software, codified into indisputably binary structures, must interact with the systems around it. The smallest, simplest system is always part of a larger one, inextricably intertwined.
Complexity and interconnectedness are tricky, but the real spanner-in-the-works for data-led decision making is humanness. People don’t behave predictably like software does (maybe the mice were on to something), a fact which renders algorithmic models inaccurate at best, misrepresentative distortions at worst. Despite this, we trust the numbers.
Metrics are useful, though. Hard data simplifies. It reduces the cognitive load during decision making, and diminishes accountability—if you do as the stats suggest, how can it be your fault when things don’t work out? Difficult decisions are tiring, and we’re generally not very good at dealing with the emotional consequences of getting them wrong. We seek to replace subjective consideration with algorithms, and to have indisputable data on hand to justify each and every commitment. Statistics are like an analeptic tonic for the ravages of decision-induced stress. But perhaps opiates make a better analogy.
We’re addicted. Dazed and confused and thinking only as far as the next hit, the next chance to measure and optimise. Desperate to find ever more opportunities to let the numbers dull the pain, we begin to lose the ability to intuit better answers. We see only the numbers. We persuade ourselves that metrics provide a more accurate picture of reality when in fact they abstract and simplify to a point of such distortion that our perception of the world becomes hopelessly skewed. Our collective grip on reality begins to slip.
Eventually we build and optimise systems that have no connection to their original purpose. We build schools that are optimised to churn out maximum grades, and economic systems optimised for profit. These things may be somewhat useful indicators, but they’re lazy objectives, constructed by our addiction to the comfort of the statistic. Easy, measurable targets.
Neither grades nor profit are broad indicators of value. Relative to the gamut of things that make up the human condition, both are microscopically specific, measuring tiny fractions of our existence. I don’t care about grades. I do care about inquisitive kids with a thirst for the world around them and a desire to contribute to the society they live in. Unfortunately, such things are impossible to accurately represent as a figure on a standardised scale, and so we divert our systems, attention and resources away from such difficult, fuzzy objectives, toward those with more easily measured outcomes.
This isn’t an anti-metrics rant. Metrics can promote better analysis and debate, and enlighten us with a greater understanding of our world and its systems. Like any science, though, findings are only useful if they’re subjected to rigorous scrutiny and applied appropriately, and when the methods for testing and measuring are improved iteratively by communities of peers who understand the flaws and potential pitfalls. Sadly, it seems that over-simplified logical fallacies—results of pseudo-scientific testing or inappropriate applications of data—often sway those with the biggest decisions to make. It’s a huge temptation, I suppose, if you’ve bought into the idea that numbers don’t lie.
I’m not sure how we break the addiction. An obvious place to start might be identifying methods and processes that exist only to improve the numbers. A solution focussed on a metric, rather than something that actually matters, often indicates an over-abstracted problem.