Polishing a Turd

turd polishWhen I was growing up, I remember my dad (a Physicist) telling me that it was pointless, and in fact meaningless, to be accurate with an estimate: if you’ve worked out a calculation using a number of assumptions, there’s no point in writing the answer to 3 decimal places! He would say that my ‘accurate’ answer would be wrong because it is misleading. The reader needs to know about the possible range of answers – i.e. about the uncertainty – so that they don’t run off thinking that it is exact.

And so, with that introduction (and flashback to my school days) this post is about the regular comedy surrounding business cases, and detailed up-front planning…and what to do instead.

A seriously important concept to start with:

The Planning fallacy

Human beings who desire something to be ‘a success’ (e.g. many an Executive/ Senior Manager) tend to:

“make decisions based on delusional optimism rather than on a rational weighting of gains, losses, and probabilities. They overestimate benefits and underestimate costs. They spin scenarios of success while overlooking the potential for mistakes and miscalculations. As a result, they pursue initiatives that are unlikely to come in on budget or on time or deliver the expected returns – or even to be completed.” (Daniel Kahneman)

This isn’t calling such individuals ‘bad people’, or even to suggest that their actions are in some way deliberate – it is simply to call out a well-known human irrationality: the planning fallacy.

We all ‘suffer from’, and would be wise to understand and guard against, it.

I’ve worked (or is that wasted time) on many a ‘detailed business case’ over the years. There is an all-too-common pattern….

“Can you just tweak that figure till it looks good…”

models are wrongLet’s say that someone in senior management (we’ll call her Theresa) wants to carry out a major organisational change that (the salesman said) will change the world as we know it!

Theresa needs permission (e.g. from the board) to make a rather large investment decision. The board want certainty as to what will happen if they sign the cheque – there’s the first problem1.

Theresa looks around for someone who can write a great story, including convincing calculations…and finds YOU.

Yep, you are now the lucky ‘spreadsheet jockey’ on this proposed (ahem) ‘transformation programme’.

You gather all sorts of data, but mainly around the following:

  • a ‘base case’ (i.e. where we are now, and what might happen if we took the ‘do nothing’ option);
  • a list of ‘improvements’ that will (supposedly) occur if the board says ‘Yes’;
  • assumptions relating to the potential costs and benefits (including their size and how/when the cash will flow); and
  • some ‘financial extras’ used to wrap up the above (interest rates, currency rates, taxes, the cost of capital…and so on)

You create an initial broad-brush model and then, after gaining feedback from ‘key’ people, you work through a number of drafts – adding in new features and much detail that they insist as being essential.

And voila! We have a beautifully crafted financial model that has a box at the end with ‘the answer’ in it2.

You show the model to Theresa.

Wow, she’s impressed with the work you’ve put in (over many weeks) and how sophisticated the model is…but she doesn’t like this initial answer. She’s disappointed – it’s not what she was looking for.

You go through all of the assumptions together. Theresa has some suggestions:

  • “I reckon the ‘base case’ comparison will be worse than that…let’s tweak it a bit”
  • “Our turnover should go up by more than that…let’s tweak it a bit”
  • “Nah, there won’t be such a negative productivity hit during implementation – the ‘learning curve’ will be much steeper!…let’s tweak it a bit”
  • “We’ll save more money than that…and avoid paying that…let’s tweak it a bit”
  • “Those savings should kick in much earlier than that…let’s tweak it a bit”
  • “We’ll be able to delay those costs a bit more than that…let’s tweak it a bit”

…and, one of my favourites:

“Mmm, the ‘time value of money’3 makes those upfront costs large compared to the benefits coming later…why don’t we extend the model out for another 5 years?”

And, because you designed a nice flexible model, all of the above ‘suggestions’ are relatively easily tweaked to flow through to the magic ‘answer’ cell

“now THAT looks more healthy! The board is going to LOVE this. Gosh, this is going to be such a success”.

Some reflections

John Dewey quote on learningSome (and perhaps all) of the tweaks might have logic to them…but for every assumption being made (supposedly) tighter:

  • one, or many, of the basic assumptions might be spectacularly wrong;
  • plenty of the assumptions are being (conveniently4) ignored for tweaking…and could equally be ‘tightened’ in the other direction (i.e. making the business case look far worse); and
  • there are many assumptions that are completely missing…because you simply don’t know about them….yet…or don’t want to know about them.

With any and every tweak made, nothing has actually changed: Nothing has been learned about what can and will actually occur. You have been ‘polishing a turd’…but, sadly, that’s not how those around you see it. Your model presents a highly convincing and desirable story.

Going back, your first high-level draft model was probably more useful! It left many ‘as-yet-unknowns’, it contained ranges of outcomes, it provided food-for-thought rather than delusional certainty.

We should reflect that “adding more upfront planning…tends to make the eventual outcome worse, not better” (Lean Enterprise). The more detailed you get then the more reliant you become on those assumptions.

The repercussions

Theresa gains approval from the board for her grand plan and now cascades the (ahem) ‘realisation of benefits’ down to her direct reports…who protest that the desired outcomes are optimistic at best, and sheer madness at worst (though they hold their tongues on this last bit).

Some of the assumptions have already proven to be incorrect – as should be expected – but it’s too late: the board approved it.

The plan is baked into cascaded KPIs…and everyone retreats into their silos, to force their part through regardless of the harm being caused.

But here’s the thing:

“Whether the project ‘succeeds’ according to [the original plan] is irrelevant and insignificant when compared to whether we actually created value for customers and for our organisations.” (Lean Enterprise)

The wider point…and what to do instead

validated learningIt’s not just financial models within business cases – it is ‘detailed up-front’ planning in general: the idea that we should create a highly detailed plan before making a decision (usually by hierarchical committee) as to whether to proceed on a major investment.

The Lean Start-up movement, led by Eric Ries, makes a great case for a totally different way of thinking:

  • assumptions aren’t true! (it seems daft to be writing that…but the existence of the planning fallacy requires me to do so);
  • we should test big assumptions as quickly as possible;
  • such testing can be done through small scale experimentation (which doesn’t require huge investment) and subsequent (open-minded) reflection;
  • we will learn important things…which we did not (and probably could not) predict through detailed up-front planning. This is a seriously good thing – we can save much time, money and pain, and create real customer value;
  • we may (and often will) find a huge flaw in our original thinking…which will enable us to ‘pivot’5 to some new hypothesis, and re-orientate us towards our customer purpose.

The big idea to get across is what has been termed ‘validated learning’.

Learning comes from actually trying things out on, and gaining direct feedback from, the end customers (or patients, citizens, employees etc.), rather than relying on our opinions about them.

Validated is about demonstrating what the customer (or patient, citizen, employee etc.) actually does (or doesn’t do), not what they say they would do when asked (i.e. from external market research or internal survey). It is to observe and measure real behaviours, rather than analyse responses to hypothetical questions.

…and to do the above rapidly by experimenting with ‘minimum viable products’ (MVPs).

Delay (whilst writing a beautiful document, getting it approved, and then building a seemingly golden ‘solution’) prevents the necessary feedback from getting through.

Caveat: Many an organisation has read ‘The Lean Startup’ book (or employed a consultant who has) and is using the above logic merely at the start of their legacy ‘investment decision’ process…but, through grafting new labels (such as Lean) onto old methods and retaining central hierarchical approval committees, their process remains ostensibly the same.

You don’t do validated learning merely at the start of an investment process – you re-imagine what ‘making investments’ means!

“It’s moving leaders from playing Caesar with their thumbs up and down on every idea to – instead – putting in the culture and the systems so that teams can move and innovate at the speed of the experimentation system.”

“The focus of each team is iterating with customers as rapidly as possible, running experiments, and then using validated learning to make real-time investment decisions about what to work on.” (Eric Ries)

 Notice that it is the team that is making the investment decisions as they go along. They are not deferring to some higher body for ‘permission’. This is made possible when:

  • the purpose of the team is clear and meaningful (i.e. based around a service or value stream);
  • they have meaningful capability measures to work with (i.e. truly knowing how they are doing against their purpose); and
  • all extrinsic motivators have been removed…so that they can focus, collaborate and gain a real sense of worth in their collective work.

Nothing new here

You might read the above and shout out:

  • “but this is just the scientific method”; or
  • “it’s yet another re-writing of the ‘Plan – Do – Study – Act’6 way of working”

…and you’d be right.

Eric Ries’ thinking came about directly from his studying of Deming, Toyota etc. and then applying the learning to his world of entrepreneurship – to become effective when investing time and money.

His book, ‘The Lean Startup’, and the ‘validated learning’ concept are an excellent addition to the existing body of work on experimentation towards purpose.

Footnotes

1. We should never present a seemingly certain picture to a board (or merely hide the caveats in footnotes)…and we should coach them to be suspicious if they see one.

2. For the financially aware: this will likely be a net present value (NPV) figure using a cost of capital (WACC) provided by the finance department, or some financial governance body.

3. The ‘time value of money’ reflects the fact that $1 now is worth more to you than $1 in a year’s time.

4. Conveniently doesn’t mean intentionally or maliciously – it can just be that lovely planning fallacy at work.

5. Pivot: This word has become trendy in many a management conversation but I think that its original (i.e.intended) meaning is excellent (as used by Eric Ries, and his mentor Steve Blank).

Eric Ries defines a pivot as “a structured course correction designed to test a new fundamental hypothesis….”

6. PDSA: Popularised by Deming, who learned it from his mentor, Walter Shewhart. A method of iterative experimentation towards your purpose, where the path is discovered as you go, rather than attempted to be planned at the start. Note that, whilst the first step is ‘Plan’, this DOESN’T mean detailed up-front planning of an answer – it simply means properly planning the next experiment (e.g. what you are going to do, how you are going to conduct it, and how you are going to meaningfully measure it).

 

A kerfuffle over Coffee

Coffee beansI was having a chat with someone about ‘batching’ the other day, and wanted to point them to a short and simple post I wrote a few years ago that I thought would assist…so I looked for it…and couldn’t find it…and then realised that I’d never published it on this blog…so here it is:


I wrote a post a bit back entitled One at a time please. In it, I attempted to explain (using my washing up at home1) about the problems caused by doing things in batches, and that we should strive to shift our processes towards ‘single piece flow’.

This is such an important point that I thought that I would put forward another, hopefully more obvious, example.

Right, here goes:

What do you see here…

Coffee - both doors closed

…yep, we had two fancy coffee machines in our last works kitchen.

If you were to watch people using them you would note that it is rare that they are both being used…but it does happen…so having 2 machines helps cope with the variation in our demand for a coffee (with spikes in demand unsurprisingly occurring Monday – Friday at around 10:30 and 15:00)

These machines need regular cleaning. I think, from my observations that this is performed weekly.

Cleaning

How about this picture?

Coffee - both doors open

Yep, this is what happened when they were both being cleaned.

We used to have a cleaner that liked to come in at around 10:30 (not such a good time really…but that’s a different story), open up both machines and then proceed to perform his cleaning steps one-by one for both machines. Something like this:

  • Take out both sets of waste drawers, empty them and put them by the sink
  • Take out both drip trays, empty and put them by the sink
  • Open up both coffee hoppers and fill them
  • Open up both creamer hoppers and fill them
  • Open up both sugar hoppers and fill them
  • Clean the pipes and connectors of both machines
  • Wash and dry the waste drawers and drip trays for both machines
  • Put everything back together for both machines
  • Close the front drawers of both machines
  • Wipe the outsides of both machines
  • …and done. Nice job.

This takes some time…and what can’t happen whilst this is being done?

No one can make a coffee! (or hot chocolate or mocha or …..name some other weird drink made from permutations of powder)

What’s the purpose of the machine? To reliably make (good) coffee as and when someone wants one.

A change in cleaner

I noticed one day that we had changed our cleaner. I also noticed this:

Coffee - one open, one closed

Oh yes! She does exactly the same steps as the earlier cleaner…but she does it one machine at a time.

Now, for those die-hard ‘economies of scale’ fans out there:

  1. There is hardly any time difference between the two cleaning approaches; BUT
  1. We can all still make coffee whilst it’s being done!!!

Even better, she concentrates on one machine at a time (in a state of flow), likely making sure that all is okay with it, potentially causing her to think far wider than just repeating a set of standard steps.

So there you go: a short and simple example of the sense in reducing batch sizes as and when we can 🙂

Anyone for a coffee?

Footnotes

1: My earlier post: If this is the first time you’ve read about batches then please do read my earlier post  – it goes into more detail.

There was one among you who, in response to this earlier washing up ‘batch to flow’ post, spent quite some time explaining to me exactly how they emptied their dishwasher, specifying how they had experimented with which items of cutlery were best held between each digits. Nice! You know who you are…Tom 🙂

2: On cleaners: I don’t know why the cleaner changed or why they adopted their different approaches and I make no judgement on either of them. I just really like being able to get a coffee whilst the machines are being cleaned!

3. Simple, and complicated: I know that there are all sorts of batches ‘out there’ (whether temporal or quantity based)…and I know that there are constraints that need to be understood and worked with (you can’t usually just remove, or even reduce, batch sizes – you have to look at the ‘why’)….but the desired target condition of ‘single piece flow’ can be used as a vision to experiment towards, whatever the nature of your batch.

Toilet Humour

UrinalSo, I’m in the process of moving office and I’m clearing out paperwork around my desk. I came across something which made me reflect, and have a giggle…and I thought I’d share:

We moved into our current building about four years ago – approximately 100 people, with only one urinal in the men’s toilet.

It all started with an email which read something like this…

“Would the men please stand over the urinal before doing their business.”

The motivation for this email? Well, let’s just say that there was a fair bit of ‘dribbling’ going on, creating ‘puddling’ on the floor…and (unsurprisingly) some of the male toilet goers weren’t particularly enamoured with their colleagues’ failure to aim…and neither was the cleaner!

So that email should have sorted it all out, yes?

No.

The next email was more direct, dropping any attempt at politeness.

Then, a hand-written sign was put above the urinal. The author’s aim was clearly to insult the culprit(s) – the phantom dribblers.

Finally, the cleaner refused to mop the male toilets.

Action was required…and this came in the form of a mop and bucket of chemical solution, bought for us men, and installed next to the urinal.

Another ‘direct and to the point’ email was sent around informing us of the ‘mop and bucket’ purchase, and what to do with it!

Things quietened down for a while. I definitely detected that a little mopping was happening. A change, but nowhere near perfection.

We then got an email about the mop. Apparently, mops left in chemical solution ‘all day, every day’ quickly dissolve – we were going through a mop head every week! We were now instructed to take the mop out of the bucket once we had cleaned up.

I observed (through my natural toilet visits each day) that the mop was being balanced in all sorts of weird positions – often not making it out of, or falling back into the bucket (splashing chemicals over the floor).

Taking a different approach

It was at this point that I saw an opportunity to experiment.

Emails operate at a point in time, far from the gemba – the urinal in this case! And those emails were clearly proving to be ineffective.

Question: When does someone most need instructions, and/or prompts?

Answer (Hypothesis): At the time and place that the ‘action’ occurs (!), in a format that they can understand and (importantly) accept1 and relate to.

Proposed Countermeasure: Create a clear (and non-judgemental) poster and put it up on the wall, by the urinal.

…and so I did:

Musings of a mop

…and every time I went to the toilet I observed the condition of the urinal2.

So how did that go?

Miraculous! The toilet floor was virtually always mopped clean (it was still shiny from the last fella passing through)…and the mop was always balanced nicely on the bucket.

Hilariously, I found myself regularly reading my own poster and checking the floor and mop3 – I was altering my own behaviours.

The floor stayed nice and clean for many months and I became bored of reading the poster (my words were annoying me)…and so I wanted to perform another experiment – what would happen if I took the poster down? And so I did.

Did the wheels fall off?

Well, no, not really.

I’d say that the floor doesn’t stay quite as clean as it used to…but, in general, people will mop up, and best of all for Mopsy, he remains nicely balanced on the side of the bucket, away from those harsh chemicals.

Learnings

In summary:

  • clear, practical and non-judgemental visual controls at the gemba really work;
  • emails telling you off, and/or telling you what to do, don’t!
  • new behaviours can become habitual (hence why the poster could be taken down4).

I know that there are loads and loads of far more meaningful examples of the enormous power of visual management…but I thought that I’d introduce a little bit of ‘toilet humour’ into our office move process…and now my fellow male office dwellers5 know who wrote the poster 🙂 .

A serious point to end

My editor for this post (thanks Paul) made the most excellent comment:

“It’s interesting that when something is not good then we behave with ‘blaming’…

Asking ‘why’ over ‘who’ is better. [Whether we do this] must be something to do with the environment!”

Paul is calling out that:

  • we so easily jump to blame, looking for the WHO…and think that by naming and shaming, we will force things to improve;
  • we can achieve so much if we look for the WHY, and then do something meaningful about this.

The emails and rude notes were aimed at people.

The mop and bucket, and visual control, were focused on the activity.

Footnotes

1. Acceptance: All those emails (and rude signs) were deficient because, just like cars, no-one believes that they are a bad driver. Unless you are presented with direct evidence of your deficient behaviour, it’s always somebody else!

2. Analysis: Don’t worry, I didn’t set up a formal measuring regime and/or start making deliberate trips to the toilet. For the benefit of any hard core researchers reading this post – my observations can only be described as anecdotal.

3. I’m not saying that I was ‘the phantom dribbler’! I just felt compelled to share the burden and have a little mop up 🙂

4. Taking down the poster: I’m not suggesting that visual controls should be taken down after a period of time. I am suggesting though, that they should be regularly revisited to be refreshed – to keep people’s attention, and improved – to make things even better.

5. …and the females around the office are either disgusted (not having known that some of the males were ‘dribblers’) or pleasantly surprised (that males can change their habits!)

’80 in 20’…erm, can we change that?!

80 in 20This is a bit of a ‘back to basics’ post, inspired by refreshing my memory from reading a superb book. It’s long…but hopefully interesting 🙂

Some years back I was working with a most excellent colleague, who managed a busy contact centre operation. Let’s call her Bob. She was absolutely committed to doing the best she could, for her staff and her customers.

Bob came to me one day for some help: Things weren’t going well, she had a meeting with senior management coming up and she was going to ask them to approve a radical thing – to change, by which I mean relax, their current call handling target.

I didn’t know too much about contact centres back then…so I started by asking some dumb questions. And it went something like this:

Me: “What’s this ‘80 in 20’ measure about?”

Bob: “It’s our main ‘Key Performance Indicator’ (KPI), called ‘Grade of Service’ (or GOS for short) and it means that we aim to pick up 80% of all incoming calls within 20 seconds of the customer calling.”

Me: “Oh…and where do these figures comes from?”

Bob: “It’s an industry recognised KPI. All ‘up to date’ contact centres use it to measure how they are doing and ‘80 in 20’ is Best Practise.”

Me: “…what ‘industry body’ and where did they get these figures?”

Bob: “The [insert name of a] ‘Contact Centre Association’…and I’ve got no idea where the figures come from.”

Me: “So, we have a target of picking up a customer’s call within an arbitrary 20 seconds…and we have an arbitrary target on meeting this target 80% of the time? …so it’s a target on a target?”

Bob: “Yes…I suppose it is…but we are having a real tough time at the moment and we hardly ever achieve it.”

Me: “Okay…but why do you want to ask senior management to ‘relax’ this target-on-a-target? What will this achieve?”

Bob: “Because we publish our GOS results against target for all our contact centre team leaders to see…and frankly there’s not much they can do about it…and this is really demoralising. If I could just get senior management to relax it to, say, 70% in 30 seconds then my staff could see that they at least achieve it sometimes.”

…and that’s how my discussion with Bob started.


I have just finished reading Donald Wheeler’s superb book ‘Understanding Variation – the key to managing chaos’ and my work with Bob1 all those years ago came flooding back to me…and so I thought I’d revisit it, and jot down the key points within. Here goes…

Confusing ‘Voice of the Customer’ and ‘Voice of the Process’

VoPI’ll start with clarifying the difference between the customer and the process. In the words of Donald Wheeler:

“The ‘voice of the customer’ defines what you want from a system.

The ‘voice of the process’ defines what you will get from a system.”

The difference in words is subtle, but in meaning is profound.

In Bob’s case, she has determined that customers want the phone to be picked up within 20 seconds2. However, this wishful thinking (a target) is completely outside the system. Bob could set the customer specification (target) at anything, but this has got nothing to do with what the process can, and will predictably3, achieve.

What we really want to see is what the system (‘handling4 customer calls’) is achieving over time.

A target is digital (on/off) – either ‘a pat on the back’ or ‘not good enough!’

On off switch “A natural consequence of this specification [target] approach…is the suddenness with which you can change from a state of bliss to a state of torment. As long as you are ‘doing okay’ there is no reason to worry, so sit back, relax, and let things take care of themselves. However, when you are in trouble, ‘don’t just stand there – do something!’ …This ‘on-again, off again’ approach is completely antithetical to continual improvement.” (Wheeler)

Unfortunately, Bob is constantly the wrong side of the (current) specification and therefore has the unwavering torment of ‘don’t just stand there – do something!’

But do what? And how would Bob know if whatever they try is actually an improvement or not? Using a target is such a blunt (and inappropriate) tool. Future results:

  • might ‘beat target’ (gaining a ‘pat on the back’) and yet simply be noise5; or
  • might still be lower than target (receiving another ‘kick’) and yet contain an important signal.

Bob cannot see the true effects of any experimentation on her system whilst relying on her current Industry best practise ‘Grade of Service’ KPI. She does not have a method to separate out potential signals from probable noise.

Thinking that a target can change things for the better

pressure“When people are pressured to meet a target value, there are three ways they can proceed:

  1. They can work to improve the system;
  2. They can distort the system; or
  3. They can distort the data.               

(Wheeler, referencing Brian Joiner)

What can a call agent do to ‘hit’ that target? Well, not much really. They can’t influence the number of calls coming in or what those customers want or need. They CAN, however, try to ‘get off the phone’ so as to get to the next call. Mmm, that’s not going to help the (customer-defined) purpose…and is probably likely to create failure demand, complaints and re-work…and make things worse.

What can the contact centre management (from team leaders and upwards to Bob) do to ‘hit’ that target? They could try to improve the system* (which, whilst being the right thing to do, is also the hardest) OR they could simply ask for the target to be relaxed. If they aren’t allowed to do either, then they might begin to ‘play games’ with the data…and hide what is actually happening.

* To improve the system, Bob needs contextual data presented such that it uncovers what is happening in the system…which will enable her to listen to the process, see signals, ask relevant questions, understand root cause, experiment and improve. She, and her team, cannot do this at present using her hugely limiting KPI.

In short, the target is doing no good…and probably some (and perhaps a lot of) harm.

It’s perhaps worth reflecting that “Bad measures = bad behaviours = bad service” (Vanguard)

What’s dafter than a target? A target on a target!

stop that its very sillyWhy? Well, because it removes us from the contextual data, stripping out the necessary understanding of variation within and thus further hiding the ‘voice of the process’.

It’s worth noting that, in Bob’s ‘20 seconds to answer’ target world:

  • A call answered in 3 seconds is worth the same as one answered in 19 seconds; and, worse
  • A call answered in 21 seconds is treated the same as one answered in, say, 480 seconds….and beyond…perhaps even an hour!

Note: I’ve added an addendum at the end of this post with a specific ‘target on a target’ example (hospital wait times). I hope that it is of use to demonstrate that using a ‘target on a target’ is to hide the important data underneath it.

“Setting goals [targets] on meeting goals is an act of desperation.” (Wheeler)

Worse still, a ‘target on a target’ can fool us into thinking that we are looking at something useful. After all, I can still graph it…so it must be good…mustn’t it?

Here’s a control chart of Bob’s ‘Grade of Service’ (GOS) KPI:

I-MR

 You might look at it and think “Wow, that looks professional with all that I-MR control charty stuff! I thought you said that we’d be foolish to use this ’80 in 20’ target on a target?”

You can see that Bob’s contact centre never met the ’80 in 20’ target-on-a-target6 (and, with the current system, isn’t likely to)…and you can perhaps see why she wants to ‘relax’ it to ’70 in 30’….but we can’t see what really happens.

What’s the variation in wait times? (times of day, days of week etc.)

Do some people get answered in 5 seconds? Is it common for some people to wait for 200 seconds? (basically, what’s actually gong on?!)

Is the variation predictable? Are there any patterns within?

Are those months really so comparable?…are any games being played?!

Okay, so I’ve shot at what Bob has before her…but what advice can I offer to help?

Does Bob need to change her ’80 in 20’ KPI?  Yes, she does….but not by relaxing the target.

‘The right data, measured right’ (‘what’, and ‘how’)

what how whyAt its very simplest, Bob’s measures need to help her (and her people) understand and improve the system.

To do this, they need to see:

WHAT matters to the customer? …which could be uncovered by:
“Don’t make me queue” Volume of calls, time taken to answer, abandonment rate.
“I want you to deal with me at my first point of contact” % of calls resolved at first point of contact (i.e. didn’t need to be passed on).
“Don’t put me on hold unnecessarily” % of calls put on hold (including reason types and frequencies).
“I want to deal with the right person (i.e. with the necessary knowledge, expertise, and authority)” % of calls passed on (including reason types and frequencies).
“I want you to action what you have promised, when I need it…and to do so first time.” Failure demand, either chasing up or complaining (including reason types and frequencies).

Now, Bob (and most contact centres) might reply “We already measure some of that stuff!”

Yes, I expect you do.

What also matters is HOW you measure it.  Measures should be:

  • shown over time, in chronological order (i.e. in control charts, to show variation), with control limits (to separate out signals from noise);

  • updated regularly (i.e. at meaningful intervals) and shown visually (on the floor, at the gemba), providing feedback to those working in the system;

  • presented/ displayed together, as a set of measures, to show the system and its interactions, rather than a ‘Grade of Service’ KPI on a dashboard;

  • monitored and analysed to identify signals, and consider the effect of each experimental change towards the customer purpose;

  • devoid of a target! The right measures, measured right will do just fine.

Why are control charts so important? Wheeler writes that:

“Instead of attempting to attach a meaning to each and every specific value of the time series, the process behaviour [i.e. control] chart concentrates on the behaviour of the underlying process.”

aeroplane dashboardWhy do we need to see a set of measures together? Simon Guilfoyle uses the excellent analogy of an aeroplane cockpit – you need to see the full set of relevant system measures to understand what is happening (speed, altitude, direction, fuel level…). There isn’t ‘One metric that matters’ and it is madness to attempt to find one.

Looking at Bob’s proposed set of capability measures (the table above), you can probably imagine why you’d want to see them all together, so as to spot any unintended consequences to changes you are experimenting with.

I.e. if one measure appears to be improving, is another one apparently worsening? Remember – it’s a system with components!

To summarise:

In a nutshellIf I am responsible for a process (a system) then I want to:

  • see the actual voice of the process;
  • get behind (and then drop) any numerical target;
  • split the noise from any signals within;
  • understand if the system is ‘in control’ (i.e. stable, predictable) or not; and
  • spot, and investigate any special causes7

and, perhaps more important, I want to:

  • understand what is causing the demand coming into the system (rather than simply treating all demand as work to be done);
  • involve all of the people in their process, through the use of visual management (done in the right way); and then
  • experiment towards improving it…safe in the knowledge that our measures will tell us whether we should adopt, adapt or abandon each proposed change.

Bob and I continued to have some great conversations 🙂


I said that I would add an addendum on the subject of ‘a target on a target’…and here it is:

Addendum: An example to illustrate the point

I’ll borrow two diagrams8 from a really interesting piece of analysis on NHS hospitals (i.e. in the UK) and their Accident and Emergency (A&E) wait times.

The first chart is of Alder Hey Children’s hospital. It shows a nice curve of the time it takes for patients to be discharged:

Alder Hey

The second chart is of Croydon University Hospital. Same type of chart, but their data tells a vastly different story!

Croydon

Q1: Do you think that an activity target has been set on the A&E system and, if so, where do you think it has been set?

I’d bet (heavily) that there is an A&E ‘time to discharge’ target, set from management above, of 4 hours (i.e. 240 minutes). It’s sort of evident from the first graph…but ‘smacks you between the eyes’ in the second.

Two further questions for you to ponder9: Looking at the charts for these two hospitals…


Q2: Which one has a smooth, relatively under control A&E system, and which do you think might be engaged in ‘playing (survival) games’ to meet the target?

I’d say that Alder Hey is doing rather well, whilst Croydon is (likely) engaged in all sorts of tricks to ship patients somewhere (anywhere!) ‘before the 4 hour buzzer’ – with a likely knock-on effect to patient experiences and outcomes;


Q3: Which one looks better on a ‘% of patients that met the 4 hour target’ league table? (i.e. a target on a target)

It is typical for health services to set an A&E ‘target on a target’ of, say, ‘95% discharged from A&E within 4 hours’10. This is just like Bob’s ‘80% in 20 seconds’.

Sadly, Croydon will sit higher up this league table (i.e. appear better) than Alder Hey!

If you don’t understand why, have a closer look at the two charts. Look specifically at the volume of patients being discharged after the 240 min. mark. Alder Hey has some, but Croydon has virtually none.

Foot notes

1. Just in case you hadn’t worked it out, she (or he) wasn’t called Bob!

2. Customer Target: Setting aside that the customer target shouldn’t (and indeed can’t) be used to improve the ‘handling calls’ system, I have two problems with the 20 second ‘customer specification’.

a. An industry figure vs. reality: rather than assuming that a generic industry figure of 20 seconds is what Bob’s customers want, I asked Bob to provide me with her call abandonment data.

I then graphed a histogram of the time (in seconds) that each customer abandoned their call and the corresponding volume of such calls. This provided us with evidence as to what exactly was happening within Bob’s system…which leads me on to:

b. An average customer vs. variety: There’s no such thing as ‘an average customer’ and we should resist thinking in this way. Some people were abandoning after a couple of seconds, others did so after waiting for two minutes. We can see that there is plenty of customer variety within – we should be thinking about how we can absorb that variety rather than meet some non-existent average.

3. Predictably, assuming that it is stable and there is no change made to the process.

4. Handling: I specifically wrote ‘handling’ and not ‘answering’. Customers don’t just want their call answered – they want their need to be met. To properly understand a system we must first set out its purpose from the customer’s perspective, and then use an appropriate set of measures that reveal the capability of the system against this customer purpose. ‘Answering calls’ may be necessary, but it’s not sufficient.

5. Noise vs. Signal: I’m assuming in this post that you understand the difference between noise and signals. If you don’t (or would like a refresh) then an earlier (foundational) posts on variation might assist: The Spice of Life

6. A clarification in respect of the example ‘I’ control chart: The Upper Control Limit (UCL) red line (at 80.55%) does not represent/ is not the 80% target. It just happens to be the case that the calculated UCL for Bob’s data works out to be nearly the same as the arbitrary target – this is an (unfortunate) fluke. A target line does not belong on a control chart!

7. Special Cause tests: The most obvious signal on a control chart can been seen when a point appears outside the upper or lower control limits. There are, however, other types of signals indicating that something special has occurred. These include ‘trends’, ‘shifts’, and ‘hugging’. Here’s a useful diagram (sourced from here):

special causes

8. Hospital charts: The full set of charts (covering 144 NHS hospitals for the period 2012-13) is here. I’ve obviously chosen hospitals at both extremes to best illustrate the point.

I can’t remember where I first came across these hospital charts – which annoys me!…so if it was via a post on your blog – I’m sorry for my crap referencing/ recognition of your efforts 🙂 

9. Here’s a 4th and final question to ponder: If, after pondering those two questions, you still think that a ‘target on a target’ makes sense then how do you cope with someone not always meeting it? Do you set them a target…to motivate them?

How about a target for the ‘target on a target’???

  • A 95% target of achieving an ‘80% of calls answered in 20 seconds’ target
  • A 90% target of achieving an ’95% of patents discharged within 4 hours’ target
  • ….

…and, if you are okay with this…but they don’t always meet it then how about setting them a target…where does the madness end?!

We are simply ‘playing with numbers’, moving ever further from reality and usefulness.

10. Hospital ‘Emergency department’ League tables:

Emergency tableHere’s a New Zealand ‘Emergency departments’ league table, ranking district health boards against each other (Source).

Notice that it shows:

  • A ‘target on a target’ (95% within 6 hrs)
  • A single quarter’s outcome
  • A binary comparison ‘with last quarter’
  • A (competitive) ranking

All of which are, ahem, ‘problematic’ (that’s me being polite 🙂

You can’t actually see how each district is performing (whether stable, getting better…or worse)

…and you certainly can’t see whether games are being played.

Inspector Clouseau

inspector-clouseauSo, a bloody good mate of mine manages a team that delivers an important public service – let’s call him Charlie. We have a weekly coffee after a challenging MAMIL 1 bike ride….and we thoroughly explore our (comedic) working lives.

And so to our most recent conversation – Charlie told me that his department is due an inspection (a public sector reality)…and how much he, ahem, ‘loves’ such things! 🙂

“Oh, and why’s that?” I say.

His response2 went something like this: “Well, they come in with a standardised checklist of stuff, perform interviews and site visits to tick off against a set of targets, and then issue a report with our score, and recommendations as to what we should be doing….it’s not exactly motivational!”

“Mmm” I say…”but presumably their intent – to understand how well things are going – is good?”

Here’s the essence of Charlie’s reply: “Yep, I ‘get’ the intent behind many of the items on their checklist, and I accept that knowing how we are doing is really important…but, rather than act like robotic examiners, I want them to:

  • understand us, and our reality (what we are having to deal with3)…and I want them to focus on what really matters in respect of our service, not hide behind narrow ‘currently trendy’ targets and initiatives set from above;

  • give us the chance to demonstrate:
    • how we are doing;
    • where we know we have room for improvement; and
    • what we are doing to get better; and finally 

  • assist us, by adding value (perhaps with useful references to what they’ve seen working elsewhere) rather than be seen as ‘taking up our time’. “

“Righto”, I said ”…that sounds excellent! I’ll write a post on that lot”…so here goes:

A ‘short but sharp’ generic critique of inspections:

inspectionInspection requires someone looking for something (whether positive or negative)…and this comes from a specification as to what they believe you should be doing and/or how you should be doing it.

The ‘compliance’ word fits here.

…and, as such, the inputs, behaviours and outcomes from inspections are rather predictable. You can expect some or all of the following:

  • people, often (usually?) from outside your service, spend time writing (often inflexible) specifications as to what you should (and should not) be doing;
  • people are employed, and trained, as inspectors of those specifications;
  • you and your team spend precious time preparing before each imminent inspection:
    • running preparatory meetings to guess what might happen;
    • window-dressing solely for the benefit of ‘the inspector’ e.g. making your work space look temporarily good, filling in (and perhaps even back dating) ‘paperwork’;
    • even performing ‘dummy runs’ (rehearsals!);
  • you and your team serenade the inspector around during their visit, with everyone on their best behaviour;
  • questions are asked, careful (guarded) answers are given, and an inspection report is issued;
  • your post-inspection time is then consumed rebutting (what you consider to be) poorly drawn recommendations and/or drafting action plans, stating what is going to be implemented to comply.

…and after it’s all over, a big sigh is let out, and you go back to how you were.

Wouldn’t it be great if, rather than playing the ‘inspection game show’, you truly welcomed someone (anyone) coming in to see what you actually do and, when they arrive, you carried on as normal because you are confident that:

  • you are operating in the way that you currently believe to be the best; and
  • you want them to see and understand this, and yet provide you with feedback that you can ponder, experiment with, and get even better at delivering against your purpose4.

…so how might you get to this wonderland?

A better way:

just-one-questionI’m a huge fan of a (deceptively) simple yet (potentially) revolutionary idea put forward by John Seddon:

“Instead of being measured on compliance, people should be assessed on whether they are able to show that they are working to understand and improve the work they do.

It is to shift from ‘extrinsic’ motivation (carrot and stick) to intrinsic motivation (pride), which is a far more powerful source of motivation.”

…and to the crux of what Seddon is suggesting:

“Inspection of performance should be concerned with asking only one question of managers:

‘What measures are you using to help you understand and improve the work?’ “

This, to me, is superb – the inspector doesn’t arrive with a detailed checklist and ‘cookie cutter’ answers to be complied with; and the manager (and his/her team) has to really think about that question!

To answer it, the team must become clear on:

  • the (true) purpose of the service that they provide;
  • what measures5 would tell them how they are doing against this purpose (i.e. their capability);
  • how they are doing against purpose (i.e. as well as knowing what to measure, they must be actively, and appropriately, measuring it for themselves);
  • what they are working on to improve, and how these are affecting the performance of their system; and
  • what fresh ideas have arisen to experiment with.

You can see that this isn’t something that is simply ‘prepared in advance’ for a point-in-time inspection. It is an ongoing, and ever maturing, endeavour – a way of working.

It means that any inspector (or interested party) can arrive at any time and explore the above in use (as opposed to it being beautifully presented in ‘this years’ audit file ring-binder)

 “Are you saying that ‘specifications’ are wrong then?”

i-love-to-clarifyBefore ending this post, I’d like to clarify that:

  • No, I’m not saying that specifications are (necessarily) wrong; and
  • I’m also not saying that scientific know-how, generated from a great deal of experience over time should be ridiculed or ignored ‘just because it comes from somewhere else’.

Taking each in turn,

  • Specifications (e.g. the current best known way to perform a task) should be owned by the team that have to perform them…and these should be:
    • of adequate depth and breadth to enable anyone and everyone to professionally perform their roles;
    • suitably flexible to cater for the variety of demands placed upon them (requiring principled guidelines rather than concrete rules); and
    • ‘living’ i.e. continuously improved as new learning occurs6

  • If there is a central function, then their role should be to:

    • understand what is working ‘out there’; and
    • effectively share that information with everyone else (thus being of great value to managers)

without dictating that a specific method should be ‘complied with’.

The point being that we should not think in terms of ‘best practise’…we should be continuously looking for, and experimenting with, better practise, suited to each scenario. This is to remove the (attempted) authority from the centre, and place it as the (necessary) responsibility where the actual work is performed, by the service (manager, and team) on the front line.

In this way, ‘the centre’ can shift itself from being seen as an interfering, bureaucratic and distant police force, to a much valued support service.

To Ponder:

If you are ‘being inspected’ then, yes, I ‘get’ that you currently need to ‘tick those boxes’…but how about thinking a little bit differently:

…how would you show those ‘inspecting you’ that you truly understand, and are improving, your system against its purpose?

You could seriously surprise them!

If you can really answer that one question, then you are likely to be operating a stable, yet continually improving service within a healthy environment, both for your team and those they serve.

Who knows – ‘the inspector’ might want to share what you are doing with everyone else 🙂

And, going back to the top – i.e. Charlie’s reply as to what he really wanted out of an inspection – I reckon he was ‘right on the money’!

A final comment…for all you private sector organisations out there:

Don’t think that this post doesn’t apply to you!

If you have centralised ‘Audit’, ‘Quality Assurance’ and/or ‘Business Performance’ teams (i.e. that are separate from the actual work), then virtually everything written above applies to your organisation.

Footnotes:

1. MAMIL: Middle aged men in lycra

2. To ‘Charlie’ – Please excuse the poetic licence that I have taken in writing the above…and I hope it may be of some use (woof woof 🙂 ).

3. What we are really dealing with: I picked the ‘Inspector Clouseau’ picture to allude to the possibility (probability?) that many an inspector, stuck with their heads in their audit checklist, hasn’t a clue about what is really going on within, and/or what really matters for, the service before them….and for some, this is still true AFTER the inspection has been completed 😦

I’m not trying to ‘shoot at’ inspectors – this is a role that you have (currently) been given. You might also like to ponder the above and thereby look to re-imagine your purpose. Doing so could dramatically improve your work satisfaction…and help improve the services that you support.

4. Purpose: not to be confused with the lottery of attempting to meet a numeric target.

5. A set of Measures that uncover how the system is performing, NOT one supposedly ‘all seeing’ KPI and an associated target.

6. Living: Years of (regularly futile) experience have proven to me that ‘learning aids’ (whether they be documents, diagrams, charts, pictures….) will only ‘live’ (i.e. improve) if they are regularly (i.e. necessarily) used by the workers to do the work. An earlier post (Déjà vu) fits here.

7. I think that a couple of previous posts are foundational and/or complimentary to this one:

The Principle of Mission: That clarification of intent, and allowing flexibility in how it is achieved, is far more important than waiting for, and slavishly carrying out ‘instructions’ from above.

Rolling, rolling, rolling: The huge, and game-changing difference between rolling out and rolling in change. One is static, the other is dynamic and purpose-seeking.

‘Chain’ beats ‘Triangle’

chain-beats-triangleFollowers of ‘Modern’ (?) management find themselves in essentially the same position: Trying to increase value1 to their investors. But, if this is their outlook, where to start?

Introducing the Triangle

Perhaps the place where ‘the triangle’ is seen most visibly is within many a traditional project management book – you will likely see a lovely little diagram within its first few foundational pages, with the words ‘quality’, ‘cost’ and ‘time’ at its points.

triangleIt will go on to suggest that the Project Manager’s job is to juggle these three variables so as to deliver ‘on time, within budget and to an acceptable quality’.

The key thing to notice is the assumption that there is an equation in which these variable are somehow related….and received wisdom goes on to suggest that there is a trade off and ‘we can’t have it all’ e.g. if we want higher quality then this would sacrifice time and/or cost.

…and the thinking behind that triangle isn’t limited to projects…it goes right across the organisation, in everything it does – basically that faster and cheaper are the opposite of higher quality.

All makes sense doesn’t it – nothing to see here. Blog post over?

Well no, as you’ve probably guessed, I’m just revving up!

And so to Dr Deming:

In his book ‘Out of the Crisis’ (1982)Dr W. Edwards Deming wrote about the “folklore…that quality and productivity are incompatible: that you can’t have both. A manager will usually tell you that it is either or. In his experience, if he pushes quality, he falls behind in production. If he pushes production, his quality suffers. This will be his experience when he knows not what quality is nor how to achieve it.”

Deming’s last line suggests that there could be value in exploring:

  • What ‘quality’ means; and
  • His thinking on how to achieve it.

What is quality?

It’s obvious isn’t it? Surely, it’s simply “how good something is!” Well, yes…but that doesn’t get us very far. It poses the rather obvious question “good for who?”

There are two levels to drill into:


Level 1 (and perhaps you’ll all be yawning reading this much-stated point) is that quality is, and can only be, defined by ‘the customer’ (or citizen or patient or….).

It follows that you can’t tell your customers what quality is, or quietly determine this for them. Instead, if you really want to deliver ‘quality’, you’d better spend time constantly understanding2 your customers and what they want/ need from your product/ service/process.


Level 2 is that there is no such thing as the average customer – no two customers are the same – and, as such, quality is defined by each unique customer…and this point has profound implications (particularly for service organisations).

For example, it would be a mistake to create a ‘customer specification’ and think that you have solved the quality conundrum. We need to understand the particular customer before us and design a system that can effectively, and efficiently, absorb their variety. This would be the opposite of trying to force them into a straight jacket.

“You’ve ‘dissed’ the triangle…but what’s your ‘Chain’ got to do with it?”

And so to Deming’s thinking on how to achieve quality. I’ll start by introducing his ‘quality chain reaction’:

Deming wrote that, in the post World War 2 period, some Japanese companies observed that “improvement of quality begets naturally and inevitably improvement of productivity.” i.e. that when quality goes up, costs actually come down. This would seem to be the opposite of our triangle!

How can this be so? Well, when the quality goes up, costs decrease due to fewer mistakes, less rework, fewer delays…reduced failure demand…and on and on. This leads to a continually improving flow.

Deming went on to write that the following “chain reaction was on the blackboard of every meeting within top management in Japan from July 1950 onwards:”

Improve quality – costs decrease – productivity improves – capture market (better quality, lower prices) – stay in business – provide jobs…and more jobs.

Notice where it ends – jobs. Contrast this with where most cost-cutting ‘initiatives’ start – jobs…but not to create them!

Deming calls out a difference in thinking3:

“Western Industry is satisfied to improve quality to a level where visible figures may shed doubt about the economic benefit of further improvement. As someone enquired, ‘how low may we go in quality without losing customers?’ This question packs a mountain of misunderstanding into a few choice words. It is typical of management’s misunderstanding in America.

In contrast, the Japanese go right ahead and improve the process without regard to figures. They thus improve productivity, decrease costs, and capture the market.” (Deming)

‘Triangle’ thinking requires a detailed business case, showing a healthy (yet imaginary) ‘return on investment’ (ROI) before anything can gain authorisation to proceed. This is, unhelpfully, labelled as ‘governance’.

‘Chain reaction’ thinking uses a clear vision, for the customer, and gets on with constantly experimenting towards it, whilst checking the results. This generates a purpose-seeking learning organisation.

Updating the Quality chain

Dan Jones, in one of his YouTube videos, expands Deming’s quality chain reaction to show its wider effect on the full organisational system4. I really liked what he had done on his slide…but I wanted to make it clearer still…and so I ‘tweaked it’ (see below5)…showing that, if you start at quality, the chain reaction is kicked off and then continues to flow around and around the system:

quality-chain-reaction

Now, many a command-and-control organisation would look at the above and shout out “that’s exactly what we are doing!”…and so, to counter this riposte, I thought I’d re-do the diagram but this time start at cost.

i.e. if your starting point is to reduce costs (usually by interrogating line items on the P&L, and focusing on activities) then you are NOT on the quality chain reaction. You would be on quite a different journey:

activity-cost-spiral

In a sentence:

Customer Purpose (which, by definition, means quality) comes first…which then delivers growth and profitability, and NOT the other way around!

…and, for all you executives/ senior managers out there, many (most!) of your people already know this6.

Footnotes

1. The definition of Value: I reflect on a rather nice quote from Jeffrey Liker: “The first question…is always: ‘What does the customer want from this process?’ This defines value.

Unfortunately, the modern corporate world has somewhat twisted this definition, and has come to believe that value is defined by the providers of ‘dead money’.

2. Understanding your customer: This requires much more than simply asking them what they want/ need. They often don’t know or, even if they do, can’t (or won’t) clearly articulate this. We need to listen to, and observe, the demands that they place on the system…and then we can truly understand how they behave.

3. Deming’s ‘Western/ American vs. Japanese’ comparison reflects the age, and focus, that he was operating within. Times have changed – not all Western/ American organisations can be tarred with the same brush…and not all Japanese organisations have stayed true to this thinking.

I suggest a modern interpretation would be to compare how organisations are run, by:

  • Command-and-Control ‘financial engineers’, attempting to use remote-control management; with
  • ‘Systems thinking’ value stream managers

4. Dan Jones presentation: See his slide with the heading ‘defining value’

5. Value for investors: I’ve added employees to the ‘value to investors’ column label within the diagram, to reiterate my recommendation that the system needs ’live moneyto enable this way of thinking.

6. A common aim: The production worker in Japan, as anywhere else in the world, always knew about this chain reaction; also that defects and faults that get into the hands of customers lose the market and cost him his job.

Once management in Japan adopted the chain reaction, everyone there from 1950 onward had one common aim, namely, quality.

With no lenders nor stockbrokers to press for dividends, this effort became an undivided bond between management and production workers.” (Deming)

I know that it’s a broken record but…this last sentence returns back to “Your Money or your Life!”

Slaughtering the ‘Sacred Cow’

moo-cowI’ve written enough posts now to ‘write a book’ 🙂 …so it’s about time I dealt with a seemingly sacred cow – the ‘Balanced Scorecard’.

Context

First, I’ll delve into a bit of history…

Robert Kaplan and David Norton performed a research project back in 1990 in respect of measuring organisational performance.

It was based on the premise that:

  • An organisation’s knowledge-based assets1 were becoming increasingly important;
  • The primary measurement system remained2 the financial accounting system; and
  • Executives and employees pay attention to what they measure and, therefore, were overly focused on the (short term) financials and insufficiently on the (longer term) intangible assets.

balanced-scorecardThe outcome of their research project was the concept of a Balanced Scorecard of measurements (and, of course, the accompanying Harvard Business School (HBS) management book).

This retained the organisation’s financial measures (as historic results) but added three additional perspectives:

  • Customer;
  • Internal Business Processes; and
  • Learning & Growth.

The last two were said to represent the lead indicators of future financial performance.

The Balanced Scorecard quickly gained traction in many corporations. This was helped by many a ‘big consultancy’ cashing in3 on the lucrative ‘implementation’ revenue stream.

Version 2.0

Over a decade later (2004) Kaplan and Norton then took things further by linking strategy formulation and execution to their measurement ideas and came up with the Strategy Map concept (and, you’ve guessed it…an accompanying HBS management book). I imagine that this was for two reasons:

1. They saw some improvements to/ holes in the original idea;

…and with my cynical hat sat jauntily on my head…

2. They now had an adoring following that would buy the sequel which, as ever, sets out:

– the big idea in detail;

– a set of carefully curated case studies; and

– instructions on how to implement ‘the big idea’ in (on?) your organisation

strategy-mapThe ‘Strategy Map’ turned the four quadrants of the balanced scorecard into a linear cause-effect view (see picture)

The idea went that the desired financial outcomes would be stated at the top, which would then be achieved by reverse engineering down the strategy map to the bottom.

Thus, through setting objectives from top down to bottom and using measures, targets and action plans (involving initiatives with business cases and budgets), the desired outcome could be achieved.

Wow, that all looks really cool – neat looking and oh-so-complete! Doesn’t it?

So why the ‘Sacred Cow’ reference?

Well, many (most?) organisations feverishly adopted the Balanced Scorecard/ Strategy Map tools and technique as if it were common sense. Indeed, some 20 years later, it has become ‘part of the management furniture’. Unquestioned…even unquestionable.

However, I believe that there are a number of serious problems within…so let’s consider whether that proverbial sacred cow deserves to be slaughtered…

There are two angles that I could come at it from:

  1. The thinking within the Balanced Scorecard/Strategy Map logic; and
  2. How organisations typically implement these ‘big ideas’.

It would be too easy to shoot at how organisations typically implement them (i.e. how they might have bastardised it4)…and you could easily accuse me of ‘cheap shots’, saying that these aren’t Kaplan and Norton’s fault. So, instead, I’ll critique the foundational logic using four headings.

Here goes…


1. Measurement:

The foundation of Kaplan and Norton’s logic is that we must have measures if we are to manage something…and this is regarded as conventional wisdom…but here’s a counter-quote from W. Edwards Deming to ponder:

“Of course visible figures are important but he that would run his company on visible figures alone will in time have neither company nor figures. The most important figures are unknown and unknowable but successful management must nevertheless take account of them.”

His point is that we seem to be obsessed with trying to measure the effect of a given change (usually to ‘claim it’ for some recognition or even reward), but that we cannot accurately do so…and it is a mistake to think that we can. Sure, we can likely determine whether a change is having a positive or negative effect on the system (and thereby try to amplify or dampen it) but we cannot isolate the change from everything else going on (internally or externally; occurring right now, previously or in the future)

Deming went on to provide some examples of ‘important but unknowable’:

  • The multiplying effect on sales that comes from a happy customer, and the opposite from an unhappy one;
  • The improvement of quality and productivity from teamwork (across the horizontal value stream and with suppliers);
  • The boost in quality and productivity all along a value stream from an improvement at any activity upstream;
  • The loss from the annual rating of people’s performance (the time taken by everyone to perform this process and, of far greater concern, the resulting de-motivation and relational damage caused)
  • …and so on

Deming famously wrote that “it is wrong to suppose that if you can’t measure it, you can’t manage it – a costly myth.”

feedback-cartoonExample: Can I manage how employees feel? Yes, by how I behave.

Should I become obsessed with measuring employee feeling through those dreaded culture surveys? No!!!!

…just continue to manage how people feel – by constantly and consistently applying simple philosophies such as the most excellent “Humanity above Bureaucracy” (Buurtzorg).

Leave the constant crappy ‘surveying of the obvious’ to those organisations that (still) don’t get it.

The balanced scorecard was derived because of the major limitations of purely financial measures. However, we should not assume that such a tool is a definitive answer for what we need to manage.

Indeed, it causes damaging behaviours – with management wearing blinkers when focusing on the scorecard “because we’ve tied all our management instruments into it and therefore that’s all that counts round here.”

The highly limited and ‘helicopter view’ scorecard becomes a major part of the ‘wrong management system’ problem.


2. Balance:

This word is used as if we need to balance our focus on the four different quadrants, playing one off against the others as if they are counterbalances to keep in check.

But this isn’t the case. If we did a little bit of, say, learning and growth (e.g. developing our people) and/or customer focus but then said “whoa…steady on, not too much…we need to balance the financials” then we aren’t understanding the nature of the system….and we certainly don’t ‘get’ cause and effect.

cause-and-effect

A metaphor for business to help explain the point:

Let’s suppose that you keep breaking out in a nasty skin rash.

You could pour ice cold water on it, apply a lotion or scratch it…until it bleeds (ouch).

These actions might appear to alleviate the effects…but they are also likely to make things worse…and none of them have considered (let alone dealt with) the cause!

If you continue to ignore the cause and just treat the (currently visible) effects, things could escalate…with new effects presenting…complicating any necessary treatments…causing long lasting or permanent damage…and even death.

If you want to get rid of the rash…and keep it that way (and perhaps even improve your skin complexion and wider health)…then you need to focus your attention on its cause:

  • are you reacting to something you are putting on your skin?
  • what about something you eat, drink or otherwise introduce into your body?
  • maybe it’s something else more complicated?

And once you’ve worked out the likely cause(s) then you need to do something about it.

You work on the cause (such as stop using that brand of sun cream or stop eating shell fish or…stop injecting heroin!!) whilst checking whether it is working by observing the effect (what the likes of Seddon and Johnson would refer to as ‘keeping the score’).

You don’t think “mmm, I’ll balance the cause and the effect”…because you understand the glaringly obvious definitions behind the words ‘cause’ and ‘effect’

Cause: A person or thing that gives rise to an action, phenomenon, or condition

Effect: A change which is a result or consequence of an action or other cause.” (Oxford Dictionary)

Okay, back to that Balanced Scorecard/Strategy map thingy and a cause – effect journey:

  • mgmt-cause-and-effectSenior Management’s beliefs and behaviours determine (i.e. cause) the management system that they choose to put into effect and (often stubbornly) retain;
  • The management system creates (i.e. causes) much of the environment that the people work within (effect);
  • The work environment is the foundation of (i.e. causes) how people act and react whilst doing their jobs (e.g. whether they are engaged, innovative, intrinsically motivated…or not);
  • How people act influences (i.e. causes) how processes are operated and the nature, size and speed of their evolution (whether by continuous or breakthrough improvements);
  • How processes operate and improve creates (i.e. causes) the outcomes that customers experience…and tell other potential customers about (i.e. as advocates or detractors);
  • Customers (whether they buy from, and advocate for us or ignore, avoid and slag us off) determine (i.e. cause) whether we stay in business.

The bl00dy obvious point is that THE FINANCIALS ARE THE EFFECT! So why are we so focused on them, other than to keep the score5.

…or, in a short, snappy sentence: This isn’t something to be BALANCED!!!!!!!!

The ‘balanced’ word keeps people tied to a ‘manage by results’ mentality, rather than managing the causes of the results such that the results then look after themselves.

What winds me up even more than the balanced bit is….wait for it…applying % weightings on the four quadrants5….usually with the financials (yes, the effect) getting the lions share!

That’s like saying “We’ll focus 75% on scratching the rash but only 25% on taking fewer heroin injections”. Aaaargh!!!

Now, you might respond to me by saying you believe that Kaplan and Norton understood the problem with the ‘balanced’ word…which is why they, ahem, ‘refreshed’ their logic with their ‘Strategy Maps’ book.

The problem with this is that they didn’t attack the results thinking, they merely added to it and, as such, many (most?) organisations continue with balancing and weighting…and spectacularly missing the point.


3. Key Performance Indicators vs. Capability:

kpi-statusOkay – let’s suppose that senior management accept that measures aren’t everything and that we shouldn’t be balancing (let alone weighting) things – I hope that we can all agree that some “right measures, measured right” (Inspector Guilfoyle) are going to be very useful…

…and so to the next whopper problem – the “measured right” bit.

Nothing (that I have seen) within the Balanced Scorecard/ Strategy Map logic reflects on, let alone deals with, the hugely important subject of variation and the need to always visualise measures over time.

Management simply use a set of KPIs on a ‘scorecard’ and look at their red down/ green up arrows against last period and/or their traffic lights against budget.

This is to completely ignore the dynamics of a system, and whether such movements are predictable or not….and therefore whether any special attention should be paid to them.

The Balanced Scorecard/Strategy Map approach can therefore create a set of Executives exhibiting the ‘God complex’ (as in “I have the answer!”) whilst being fooled by randomness” (Taleb) – blissfully ignorant of the capability of their value streams (or processes within) and doing much damage by tampering.


and last, but by no means least…

4. Strategy vs. Purpose:

The underlying assumptions within the Balanced Scorecard/Strategy Map thinking would appear to be the conventional ‘shareholder value’ view of the world.

(I’ve previously written a 5-part serialised post on what I think about this….so I won’t repeat this here)

We get fed a feast of:

In short: The core problem (for me) with Kaplan and Norton’s two books is that, not only do they retain the problematic traditional command and control management system, focused on delivering shareholder value – they use it as their foundation to build upon.

It’s therefore no wonder that organisations carry on as before (doing the same crappy stuff), whilst waving their supposedly game-changing ‘Strategy Map’ around a lot.

Have you got hold of that cow? Good…now where’s my ceremonial knife?


To end: ‘having a go’ at me because I’m being so negative

You might shout back “okay you cynic…what would you do instead?!”

Well, I’m not going to be able to answer that in a paragraph – even Kaplan and Norton took two (rather verbose) books…and more than a decade in-between…to present their logic – but I’d suggest that, if you are curious, the 130+ posts on this site would go some way to expressing what I (and I believe my giants) think.

…and if you want to start at measurement then you might want to look here first.

Footnotes:

1. Knowledge based assets: Kaplan and Norton list the following as examples of assets that aren’t measured and managed by financial measures: employee capabilities, databases, information systems, customer relationships, quality, responsive processes, innovative products and services.

2. Measurement system remaining financially based: H. Thomas Johnson’s book ‘Relevance Regained’ makes clear that it wasn’t always so. Financial measures used as operational measures (a bad idea) only came into being from the 1950s onwards. Johnson refers to the period 1950s – 1980s as the ‘Dark Age of Relevance Lost’ and ‘Management by Remote Control’. I would argue that many an organisation hasn’t exited this period.

3. Big consultancies ‘cashing in’: I can (sadly) write this because I have first hand evidence – I was there! 😦

4. Bastardising the Strategy Map includes organisations changing the order of the four elements!!!

5. Financials: There’s a HUGE difference between a) using financial measures to keep the score (which would be good governance) and b) attempting to use them to make operational decisions! Using financials to make operational decisions is to attempt to ‘make the tail wag the dog’.

Yes, accountants should keep the score, for cash flow monitoring and assisting with longer term investment decisions…but accountants should not be attempting ‘remote control management’ of operations.

6. Weighting the elements of the scorecard: See, for example, fig. 9.8 in ‘The Balanced Scorecard’ (1996) and the related commentary.

7. Diversity: I understand that the cow is a holy animal to some. Please don’t be offended by my use of an English phrase in expressing my thinking – no real cows were harmed in the writing of this post…and no harm is intended to those living now, or in the future 🙂