Polishing a Turd

turd polishWhen I was growing up, I remember my dad (a Physicist) telling me that it was pointless, and in fact meaningless, to be accurate with an estimate: if you’ve worked out a calculation using a number of assumptions, there’s no point in writing the answer to 3 decimal places! He would say that my ‘accurate’ answer would be wrong because it is misleading. The reader needs to know about the possible range of answers – i.e. about the uncertainty – so that they don’t run off thinking that it is exact.

And so, with that introduction (and flashback to my school days) this post is about the regular comedy surrounding business cases, and detailed up-front planning…and what to do instead.

A seriously important concept to start with:

The Planning fallacy

Human beings who desire something to be ‘a success’ (e.g. many an Executive/ Senior Manager) tend to:

“make decisions based on delusional optimism rather than on a rational weighting of gains, losses, and probabilities. They overestimate benefits and underestimate costs. They spin scenarios of success while overlooking the potential for mistakes and miscalculations. As a result, they pursue initiatives that are unlikely to come in on budget or on time or deliver the expected returns – or even to be completed.” (Daniel Kahneman)

This isn’t calling such individuals ‘bad people’, or even to suggest that their actions are in some way deliberate – it is simply to call out a well-known human irrationality: the planning fallacy.

We all ‘suffer from’, and would be wise to understand and guard against, it.

I’ve worked (or is that wasted time) on many a ‘detailed business case’ over the years. There is an all-too-common pattern….

“Can you just tweak that figure till it looks good…”

models are wrongLet’s say that someone in senior management (we’ll call her Theresa) wants to carry out a major organisational change that (the salesman said) will change the world as we know it!

Theresa needs permission (e.g. from the board) to make a rather large investment decision. The board want certainty as to what will happen if they sign the cheque – there’s the first problem1.

Theresa looks around for someone who can write a great story, including convincing calculations…and finds YOU.

Yep, you are now the lucky ‘spreadsheet jockey’ on this proposed (ahem) ‘transformation programme’.

You gather all sorts of data, but mainly around the following:

  • a ‘base case’ (i.e. where we are now, and what might happen if we took the ‘do nothing’ option);
  • a list of ‘improvements’ that will (supposedly) occur if the board says ‘Yes’;
  • assumptions relating to the potential costs and benefits (including their size and how/when the cash will flow); and
  • some ‘financial extras’ used to wrap up the above (interest rates, currency rates, taxes, the cost of capital…and so on)

You create an initial broad-brush model and then, after gaining feedback from ‘key’ people, you work through a number of drafts – adding in new features and much detail that they insist as being essential.

And voila! We have a beautifully crafted financial model that has a box at the end with ‘the answer’ in it2.

You show the model to Theresa.

Wow, she’s impressed with the work you’ve put in (over many weeks) and how sophisticated the model is…but she doesn’t like this initial answer. She’s disappointed – it’s not what she was looking for.

You go through all of the assumptions together. Theresa has some suggestions:

  • “I reckon the ‘base case’ comparison will be worse than that…let’s tweak it a bit”
  • “Our turnover should go up by more than that…let’s tweak it a bit”
  • “Nah, there won’t be such a negative productivity hit during implementation – the ‘learning curve’ will be much steeper!…let’s tweak it a bit”
  • “We’ll save more money than that…and avoid paying that…let’s tweak it a bit”
  • “Those savings should kick in much earlier than that…let’s tweak it a bit”
  • “We’ll be able to delay those costs a bit more than that…let’s tweak it a bit”

…and, one of my favourites:

“Mmm, the ‘time value of money’3 makes those upfront costs large compared to the benefits coming later…why don’t we extend the model out for another 5 years?”

And, because you designed a nice flexible model, all of the above ‘suggestions’ are relatively easily tweaked to flow through to the magic ‘answer’ cell

“now THAT looks more healthy! The board is going to LOVE this. Gosh, this is going to be such a success”.

Some reflections

John Dewey quote on learningSome (and perhaps all) of the tweaks might have logic to them…but for every assumption being made (supposedly) tighter:

  • one, or many, of the basic assumptions might be spectacularly wrong;
  • plenty of the assumptions are being (conveniently4) ignored for tweaking…and could equally be ‘tightened’ in the other direction (i.e. making the business case look far worse); and
  • there are many assumptions that are completely missing…because you simply don’t know about them….yet…or don’t want to know about them.

With any and every tweak made, nothing has actually changed: Nothing has been learned about what can and will actually occur. You have been ‘polishing a turd’…but, sadly, that’s not how those around you see it. Your model presents a highly convincing and desirable story.

Going back, your first high-level draft model was probably more useful! It left many ‘as-yet-unknowns’, it contained ranges of outcomes, it provided food-for-thought rather than delusional certainty.

We should reflect that “adding more upfront planning…tends to make the eventual outcome worse, not better” (Lean Enterprise). The more detailed you get then the more reliant you become on those assumptions.

The repercussions

Theresa gains approval from the board for her grand plan and now cascades the (ahem) ‘realisation of benefits’ down to her direct reports…who protest that the desired outcomes are optimistic at best, and sheer madness at worst (though they hold their tongues on this last bit).

Some of the assumptions have already proven to be incorrect – as should be expected – but it’s too late: the board approved it.

The plan is baked into cascaded KPIs…and everyone retreats into their silos, to force their part through regardless of the harm being caused.

But here’s the thing:

“Whether the project ‘succeeds’ according to [the original plan] is irrelevant and insignificant when compared to whether we actually created value for customers and for our organisations.” (Lean Enterprise)

The wider point…and what to do instead

validated learningIt’s not just financial models within business cases – it is ‘detailed up-front’ planning in general: the idea that we should create a highly detailed plan before making a decision (usually by hierarchical committee) as to whether to proceed on a major investment.

The Lean Start-up movement, led by Eric Ries, makes a great case for a totally different way of thinking:

  • assumptions aren’t true! (it seems daft to be writing that…but the existence of the planning fallacy requires me to do so);
  • we should test big assumptions as quickly as possible;
  • such testing can be done through small scale experimentation (which doesn’t require huge investment) and subsequent (open-minded) reflection;
  • we will learn important things…which we did not (and probably could not) predict through detailed up-front planning. This is a seriously good thing – we can save much time, money and pain, and create real customer value;
  • we may (and often will) find a huge flaw in our original thinking…which will enable us to ‘pivot’5 to some new hypothesis, and re-orientate us towards our customer purpose.

The big idea to get across is what has been termed ‘validated learning’.

Learning comes from actually trying things out on, and gaining direct feedback from, the end customers (or patients, citizens, employees etc.), rather than relying on our opinions about them.

Validated is about demonstrating what the customer (or patient, citizen, employee etc.) actually does (or doesn’t do), not what they say they would do when asked (i.e. from external market research or internal survey). It is to observe and measure real behaviours, rather than analyse responses to hypothetical questions.

…and to do the above rapidly by experimenting with ‘minimum viable products’ (MVPs).

Delay (whilst writing a beautiful document, getting it approved, and then building a seemingly golden ‘solution’) prevents the necessary feedback from getting through.

Caveat: Many an organisation has read ‘The Lean Startup’ book (or employed a consultant who has) and is using the above logic merely at the start of their legacy ‘investment decision’ process…but, through grafting new labels (such as Lean) onto old methods and retaining central hierarchical approval committees, their process remains ostensibly the same.

You don’t do validated learning merely at the start of an investment process – you re-imagine what ‘making investments’ means!

“It’s moving leaders from playing Caesar with their thumbs up and down on every idea to – instead – putting in the culture and the systems so that teams can move and innovate at the speed of the experimentation system.”

“The focus of each team is iterating with customers as rapidly as possible, running experiments, and then using validated learning to make real-time investment decisions about what to work on.” (Eric Ries)

 Notice that it is the team that is making the investment decisions as they go along. They are not deferring to some higher body for ‘permission’. This is made possible when:

  • the purpose of the team is clear and meaningful (i.e. based around a service or value stream);
  • they have meaningful capability measures to work with (i.e. truly knowing how they are doing against their purpose); and
  • all extrinsic motivators have been removed…so that they can focus, collaborate and gain a real sense of worth in their collective work.

Nothing new here

You might read the above and shout out:

  • “but this is just the scientific method”; or
  • “it’s yet another re-writing of the ‘Plan – Do – Study – Act’6 way of working”

…and you’d be right.

Eric Ries’ thinking came about directly from his studying of Deming, Toyota etc. and then applying the learning to his world of entrepreneurship – to become effective when investing time and money.

His book, ‘The Lean Startup’, and the ‘validated learning’ concept are an excellent addition to the existing body of work on experimentation towards purpose.

Footnotes

1. We should never present a seemingly certain picture to a board (or merely hide the caveats in footnotes)…and we should coach them to be suspicious if they see one.

2. For the financially aware: this will likely be a net present value (NPV) figure using a cost of capital (WACC) provided by the finance department, or some financial governance body.

3. The ‘time value of money’ reflects the fact that $1 now is worth more to you than $1 in a year’s time.

4. Conveniently doesn’t mean intentionally or maliciously – it can just be that lovely planning fallacy at work.

5. Pivot: This word has become trendy in many a management conversation but I think that its original (i.e.intended) meaning is excellent (as used by Eric Ries, and his mentor Steve Blank).

Eric Ries defines a pivot as “a structured course correction designed to test a new fundamental hypothesis….”

6. PDSA: Popularised by Deming, who learned it from his mentor, Walter Shewhart. A method of iterative experimentation towards your purpose, where the path is discovered as you go, rather than attempted to be planned at the start. Note that, whilst the first step is ‘Plan’, this DOESN’T mean detailed up-front planning of an answer – it simply means properly planning the next experiment (e.g. what you are going to do, how you are going to conduct it, and how you are going to meaningfully measure it).

 

“Sir, Sir, Sir…have you marked it yet?!”

class with hands upSo my son had some school exams and this post was triggered from a conversation I had with him just afterwards:

I expect all of you can cast your minds back to school and if you’ve got teenagers then, like me, you will also be sharing their experiences.

Picture the following scenario:

  • You’ve studied for, let’s say, a maths exam1;
  • You’ve spent 2 long hours sat on an uncomfortable school chair, whilst being watched by the beady eyes of the maths teacher (who was actually asleep), and have just emerged from the exam hall;
  • You and your mates fall straight into discussing the trauma that you’ve just been through:

“What did you put for question 4?”

“Oh [beep], I hadn’t realised it was about that! I wrote about [something else that was completely irrelevant to the question]”

“Could you work out the pattern in that sequence of numbers?…’Fibonacci’ who?”

“What do you mean there were more questions over the page?!!!”

…and so on.

What you will notice is that they are all ‘switched on’ in the moment, whether they ‘enjoyed’ the exam or not. They really want to know what the answers were and how they did against them!

The after’math’ 🙂

So, next day, they have double-maths…whoopee!

The Students all plead together: “Sir, Sir, Sir…have you marked our exam yet?”

Teacher: “Whoa, hold your horses, I’ve barely sat down! I’ll do it as soon as I can.”

…and the students engage in yet more chatter about the exam but their memory of the exam is beginning to fade.

At the end of the week, they have maths again:

The majority of Students: “Sir, Sir, Sir…have you marked our exam yet?”

Teacher: “No, not yet, I’ll do it over the weekend.”

…much less chatter now. They have forgotten most of it.

So, now it’s the following week and maths:

A few keen Students: “Sir, have you marked our exam yet?”

Teacher: “Sorry, no, I’ve been writing reports so I haven’t got around to it yet. I’ll definitely do it by the end of this week.”

…the mood has changed. The content of the exam has been forgotten and so, instead, they fall back to merely wanting to know a score.

End of week 2 maths lesson:

One diligent Student: “Sir, have you marked our exam yet?”

Teacher: “Yes I have! I’ll read out the marks” and the marks are duly read out to the class, which brings out the whole spectrum of emotions (from feelings of elation to tears of despair, with a healthy dose of indifference in between).

That diligent Student again: “…but Sir, can I have my marked exam paper back?”

Teacher: “Erm, yes…I haven’t got them with me now…I’ll bring them in next week.”

What do we think about this?

We all know that by far the best thing to do for effective learning to take place is to mark this exam, get the marked papers back to the students and then go through the paper to explain and then discuss it question-by-question…and to do all of this As Soon As Possible.

(… and I know that this is what all good teachers will try to do)

We can see that:

  • There is a human desire for immediate and meaningful feedback, which quickly dissipates over time;
  • An overall score (the result), whilst potentially providing some useful indicative data, cannot help with learning – you can feel emotions from receiving a score but you can’t improve. Instead, you need to know about the method (or, in this exam scenario, each question);

“We don’t learn from our mistakes, we learn from thinking about our mistakes” (Ralph Tyler, Educator)

  • There is little point in just the teacher knowing the current capability of each of their students. Each student should be very clear on this for themselves.

So, to organisations:

The above might seem blindingly obvious and a world away from work but every day we all carry out actions and interactions within value-streams for the good of our customers…and the usual buzz phrase uttered at regular intervals is ‘we want to continuously improve!‘…but do we provide ourselves with what we need to do so?

Think of the richly varied units of customer demand that we* strive to satisfy as analogous to the maths exam:

  • (how) do we all know how we (really) did?
  • (how) do we find this out quickly?
  • (how) do we know what specifically went well and what didn’t?
  • …and thus, (how) can we learn where to experiment and how this went?!

(* where ‘we’ refers to the complete team along the horizontal value stream)

There’s not much point in senior managers receiving a report at the end of the month that provides them with activity measures against targets and some misleading up/down arrows or traffic light colouring. Very little learning is going to occur from this…and, worse, perhaps quite a bit of damage!

…and when I say learning, I hope you understand that I am referring to meaningful changes being made that improve the effectiveness of the value stream at the gemba.

The value-creating people ‘at the gemba’:

The people who need the (relevant) measures are the people who manage and perform the work with, and for, the customer.

If the people who do the work don’t know how they are truly doing from the customer’s point of view then they are no different from the students who don’t have their marked exam papers back.

hamster wheelThere should be no surprise if the workers are merely clocking in, turning the wheel, collecting their pay and going home again. It’s what people end up doing when they are kept in the dark….though they likely didn’t start out like this!

Senior Management may respond with “but we regularly hold meetings/ send out communications to share our financial results with them, and how they are doing against budget!”

  • This gives people the wrong message! If you lead with, and constantly point at, the financials, you are telling people that the purpose of the system is profit, and NOT your stated ‘customer centric’ purpose;
  • You can’t manage by financial results. This is an outcome – ‘read only’. You have to look at the causes of the results – the operational measures;

To repeat a hugely important John Seddon quote:

“Use operational measures to manage, and financial measures to keep the score”

I am championing what may be termed as ‘visual management’: being able to easily see and understand what is happening, in customer terms, where the work is done.

A whopping big caution

caution signHowever, ‘visual management’ should have a whopping big warning message plastered all over its box, that people would have to read before undoing the clasps and pushing back the lid…because visual management works for whatever you put up on the wall!

If you put up a visual display of how many calls are waiting or how long your current call has taken or a league table of how many sales each member of your team has made or….etc. etc. etc. people WILL see it and WILL react….and you won’t like the dysfunctional behaviours that they feel compelled to engage in!

So, rather than posting activity measures and people’s performance comparisons, what do the value creating people need to know? Well, put simply, they need to know how their system is operating over time, towards its purpose.

Here’s what John Seddon says about the operational measures that should be “integrated with the work: In other words they must be in the hands of the people who do the work. This is a prerequisite for the development of knowledge and, hence, improvement.

  • Flow: what is the capability of the system to handle demands in one-stop transactions? Where a customer demand needs to go through a flow, what is the capability of that flow, measured in customer terms? 

…in both cases we need to know the extent of variation – by revealing variation we invite questioning of its causes. By acting on2 the causes, we improve performance.”

A final thought: This blog has often said “don’t copy manufacturing because Service is different! But gemba walks through a well run ‘Lean thinking’ factory floor may very well assist your understanding of what is meant by good visual management. No, I’m not saying ‘copy what you see’…I’m suggesting that you might understand how a well run value stream has a physical place alongside the gemba where its participants gather and collaborate against a background of what they are currently achieving (their current condition) and what experiments they are working on to improve towards some future target condition.

To close – A shameless segue:

So I’ve been writing this blog for nearly 2 years…and I know many people read it…but I don’t get much feedback3.

If you have read, and accept the thinking within this post, you will understand that this limited feedback ensures that I am somewhat ‘in the dark’ as to how useful my writings are for you.

I do know that people see/ open my posts…but I don’t know too much more:

  • you might read the title or first few lines of a post, yawn, and go and do something else;
  • you might get half way through and not understand what on earth I am rambling on about;
  • you might read to the end and violently disagree with some or all of what I’ve written;

but…and here’s the punch line, how would I know? 🙂

Notes:

  1. It’s clearly a totally separate, and MUCH bigger question as to whether taking exams is good for learning – I’m aware that many educators think otherwise. The genesis of this post merely comes from my son’s exam reality. Just for clarity: I’m not a fan of the ‘top-down standards and constant testing’ movement.  
  2. Seddon writes ‘acting on’, NOT ‘removing’ the causes of variation. The aim is not to standardise demand in a service offering…because you will fail: the customer comes in ‘customer shaped’. The aim is to understand each customer’s nominal value and absorb it within your system as best you can…and continue to experiment with, and improve how you can do this.
  3. A big thanks to those of you that do provide me with feedback!….and I’m most definitely not criticising those that don’t comment – I’m just saying that I have a very limited view on how I am performing against my purpose…just like many (most?) people within their daily work lives.