Gartner used this phrase extensively in support of its value proposition about the need for comparative data (Although much is made of having data, the underpinning value proposition is founded on the reputation of the company that backs the assertion of the relevant axes of goodness. In other words, it’s not just that they have the data, it’s that they have established which data is worth measuring and comparing).
If you cannot objectively tell if something has changed it’s pretty much impossible to rationally make any causal connection between your planned behaviour and expected outcome, let alone demonstrate it afterwards.
But the title of this section has (in different forms) been attributed to others – notably Deming and Drucker[1]. Deming said “you can’t manage what you can’t measure” – but (crucially) this is part of a longer quotation: It is wrong to suppose that if you can’t measure it, you can’t manage it – a costly myth.”[2]
My understanding of Demings writing includes two concepts. First – he asserts that there are things you cannot measure but still need to manage[3]. Second – he includes a wealth of example measures (all too familiar to anyone with experience in public or private sectors) that he positions as having a strong negative impact on the management of the organisation in question.
A checklist of questions for those considering measurement:
- Is the thing you are interested in measurable? Is there a reference scale? What form does it take (see NOIR scale )? Is “better” clearly defined everywhere?
- If not directly measurable, is there a proxy for what we are interested in that behaves similarly?[4]
- What behaviour changes can we anticipate if we were to establish and report on this measure?
Management is broader than measurement. Systematic improvement does require systematic measurement. Care needs to be taken when determining your measures (see also you get what you measure)
[1] Often mentioned but I haven’t been able to trace the source. Often also states as “You cannot improve what you can’t measure – which is a more defensible proposition).
[2] Deming in book The New Economics (p35 in my copy)
[3] For me this is a bit too nuanced but I certainly agree that there are things that need to be managed where an intuitive (but naive) attempt at measurement is counter-productive. Management requires a broader and sometimes softer set of information feeds that Deming might not count as “measurement”.
[4] A similar concept to Real Options Pricing