Fat Boy Slim Boy

Watch your scale | Metrics are overrated

Nov 13, 2020

About | Previously on...

Hello,

Although I like to write about boring stuff related to work, organizations, markets, and so on, the idea of landing on your inbox is that we can take it personally.

You know, it is much easier to create empathy if we let our boundaries permeate and show some vulnerability.

I don’t do this lightly. My friends know I don’t usually like to talk about myself, and the worldwide web knows I hardly publish any personal stuff on social media.

But ok, let’s go.

I decided to hire a personal trainer.

I’m not sure why but I remember that was after the summer holidays that I decided to lose some weight. I went on a diet and became more mindful of what I ate. Later I subscribed to some online training programs and started to exercise.

In October, I got a cold, and for two weeks, I took a break from that regimen. When it was time to get back to it, I admitted that the online training program was not really working — I didn’t have enough willpower to follow that program. Looking back now, I was not pushing myself enough to have any meaningful impact. So, I decided that I needed professional help.

Yes, I’m paying to suffer, to feel pain, to be embarrassed by my lack of strength. Humans do strange things…

Losing weight

Many of us have (or had) the objective of losing weight.

We finally decide to go to the gym and turn that gym subsidy into something we use. Maybe we hire a personal trainer or subscribe to an online exercise program.

We commit to being more mindful of what we eat and make an effort to cut those carbs and that late-night craving.

To keep track of our objective of losing weight, we check the scale every day. In the first week, we see some improvements! Great! All the efforts are paying off.

But then, after 2 weeks, progress stalls. We think it's probably a technical issue that the scale barely moves. We try it with your clothes on, and it moves up, we drag the dog (or kid) to the scale, and it moves down. All good 😢

You think you probably just need to push harder. You double down on exercise, excommunicate sugar, minimize carbs even further, and watch for sugary fruits.

After 1 month, the scale stubbornly continues to barely give any sign of life. You're losing patience. So much suffering and no “return on scale”!

Your leading indicators are all there — you follow the diet and exercise every day. You're feeling better, more agile, leaner, but the scale still doesn't move.

What's happening?

The metric we choose - bodyweight - contradicts our perception that things are moving in the right direction.

Do we have a problem with the measure or the goal?

When we say you want to reduce weight, most of us think we want to become slimmer. We're using bodyweight as a proxy, a metric for slim. What we probably want is to reduce body fat, those parts of your body that you don't quite like, miss-placed curves. Maybe it's in your belly or butt.

I would even say that unless you're in competitive boxing or some other sport that uses bodyweight to establish categories, body weight is not the goal. What you want is:

to be more agile, stronger
improve your body shape
or just make sure you're healthier

But you can’t easily measure agility — usually, we just feel it.

Health is also not easy to measure — you just believe that this regimen will make you healthier over time. At least it is what you read or what your doctor told you.

But yes, we can measure body shape — maybe you want to track your waist or hip, or better the waist-hip ratio.

Going back to our regimen, you know what's happening here: you're probably losing body fat but gaining body muscle due to the intense exercise. Although you feel much better, and like what you see in the mirror, the scale has been blind to this transformation.

You're measuring the wrong thing for your real objective.

The reverse can also happen: maybe you're losing weight, but you're mostly losing body water instead of body fat — and that will probably not get you where you want in terms of that idealized body shape.

The bottom line is:

it's fine to pick a metric, maybe you need a set of metrics, but keep your eyes open and check if the metric is showing the progress you're seeing.
A measure gives you a number, but most of the time you already know if it's better or not. You can recognize the patterns, you sense it.
And things don’t fit, question your goal and ask why — is that really what you want to improve?

M&M 1 — Metrics & Management

Peter Drucker supposedly said:

you can't manage what you can't measure

Makes sense. If there are things you want to improve, without measurement, you can't spot if you're improving or not. How are you going to find out what's working or not? How do you define success? How do you evaluate different improvement alternatives?

Modern management, as a discipline, really started during the Industrial Revolution. Naturally, it looked at an organization as a machine, and those commanding it were concerned with inputs, outputs, measurements, and controls. This heritage is still very much alive, and you can see it all around the metrics orthodoxy.

M&M 2 — Metric as a Map

Measuring enables us to create a map, a representation of reality. Say you measure some quality metric of output, and you want to maximize that, under some constraints. But what you want is not really to improve quality. What you're looking for is to make customers happy with your product. So you’re using Quality as a proxy to the satisfaction that is also a proxy for re-purchase and recommendation that, in the end, will improve your sales and market share.

The issue is that these proxies are not linear causalities: if the quality is great → , customers will be happy.

What we’re missing is that while product quality is a necessary condition for customer satisfaction, it is not a sufficient condition. There can be other factors that the customer also values. Maybe customer happiness is found when we meet simultaneously Quality, Factor X, and Factor Y. And maximizing Quality per se can lead to a deterioration of X or Y, which can lead to missing the heart of the customer.

So every measure is a proxy to what you want to achieve. Every metric is a map of a territory. And while having a map is a great help to explore the territory or find a particular place, it's dangerous to blindly follow the map without scanning the territory and seeing where you step into. Just imagine following the shortest path in a mountainous region with a map that doesn’t show elevations.

M&M 3 — Metrics and Models

When you have a metric and a goal, you’re assuming that you have a model that informs your plan to get to the desired goal. In our initial example, your model is:

increase exercise + decrease calories intake => lose weight
or to lose weight, I will increase my exercise and decrease the calories I eat.

And this shows that having a metric is not sufficient. You need a model that translates inputs into outputs. So all the trouble of measuring can be jeopardized if you’re using the wrong model. This happens more often than not as many of us fall into a linear causal model that neglects feedback loops and other system dynamics concepts.

What gets measured becomes commoditized

What if everyone can measure and model the metric? What stops everyone from reaching the same point B?

Of course, there’s intellectual property and other unique capabilities that can get you ahead. But it seems to me that most of the time, this leads either to marginal improvements or they are specific to a particular industry.

For example, every retailer knows how to measure and model the cashier's speed and efficiency, the optimal product assortment, or location. So, what sets them apart? What is the metric that the leader is maximizing more than others?

It is not about maximizing sales. That's a result — a lagging indicator. Following Drucker: what is the measure that the manager should manage to achieve success?

What if what’s get measured gets commoditized?
What if the non-measurable qualities are what creates the space for differentiation and great experience?

We can argue that the most important and differentiable things are not easily measurable and hard to model:

commitment
kindness
care
respect
empathy
bonding
creativity
excitement
trust
reputation

You can try to measure them with some surveys - but they will give you just a very rough "map" of what's happening.

And even if you can measure any of them, you still need a good model to influence them. Good luck!

Take the case of Marketing. We can agree that Marketing should drive revenue. You invest in it and expect a return. Fair.

The problem is, how do you measure that. And I'm not referring to the challenge of attribution. It's a much bigger challenge:

If Marketing is about changing perceptions and driving a change of behaviors, then that takes time. First, this delta between action → reaction (marketing → purchase) creates a typical control-with-delay problem. Adjusting your execution/spending to the outcome is quite hard unless your market has a superfast fast purchase cycle.

Second, this time delta creates a space for many other interferences in the process. Many other things can happen between exposure and purchase — they read a comment or news, talked to a current customer, meet your sales guy, or things just happen outside your control... This kills any hope of causality and predictability.

Marketing touches many different stages of the flow from attraction to customer conversion — from the brand and messaging to sales support materials and many things in between. How do you know what’s working? E.g., if the sales team starts to close more deals, is that because they got better at it or because the materials and positioning used are more effective? Or is that we’re attracting prospects that have a better fit?
Many Marketing activities take time to compound and have an effect — brand and content marketing are two prime examples. This means that your investment curve is probably quite different from your results curve:

Your investment may follow a logarithmic trend (red) — it grows fast at the beginning with the urge to impact the business, but it stalls waiting for the results to come in.
Your results probably follow an exponential trend (green):
- They start small because you didn't invest enough, for enough time, to get results.
- But then the effects start to compound and accelerate.
So if you’re tracking ROI on a quarter to quarter basis, you’re going to have some difficult quarters to explain, and you’ll give up before you reach any breakeven point.

What now?

We know how to manage things that we can measure, but how do we manage things that we can't measure? Here are some suggestions:

Track leading indicators — what can you measure that might lead to the results you actually want.
Raise your head and stop fixating on the “map”. When you can't blindly trust the metric (the map), you need to look up to understand better the territory: what's going on, when actually to adjust, or change courses.
Use observation and inquiry to probe what’s happening and why.
Use your intuition or gut — if you have some experience, you can rely on your intuition to spot when something doesn’t feel right so you can dive deeper and understand what’s happening.
Directionally correct — sometimes, you don’t need a precise measure. You just need to know that you're going in the right direction.
Accept the fact that a lot of what we do is based on belief — that doesn’t sound very professional, but you just need to accept that this is already happening today. You need to believe that you’re on an “exponential curve” and things will pick up.
But belief is just a necessary condition — you can’t succeed only with it. You still need to put on the hard work and keep your senses on alert.

Finally, metrics are a necessary but not sufficient condition for good management:

Managing without metrics is dangerous.
Managing only looking at metrics is suicidal.

#worth_reading

This week's theme is the free energy principle. I had never heard of it before until a friend shared the following article this week. It tells you how living beings exist despite apparently contradicting the 2nd law of thermodynamics, maximizing entropy. This happens because they are minimizing free energy, and this means minimizing surprise.

The Genius Neuroscientist Who Might Hold the Key to True AI
A fascinating article about free-energy thinking, his author Karl Friston, and the impacts on how we understand the brain and how we design AI systems.

Then I did a bit more digging and found Karl Friston himself explaining the principle in a well worth it 15 min video — amazing.

Music break

If you reach this point, you deserve a music break. This one is inspired by the body-shape conversation before 🙃

I’m sure you don’t agree with all of the above. I would love to hear about that.

Just hit reply or use the comments.

And if I got your attention till now, I honestly deserve a ❤️

Thanks for reading,

Hugo

Slightly Maverick