the dark side of removing complexity

Complexity is often public enemy number 1 for technologists in large corporations, and rightly so.  Lets forget about good versus bad complexity, and focus on the latter.  Okay, so I often use other peoples quotes, but here is an original (as far as I know):

“Complexity is highly resilient and therefore difficult to remove, but it is most happy to move, typically into somebody else’s area.” 

Think about the times you have seen complexity being removed, and about the flow on effects from a holistic viewpoint.  Was the complexity actually removed?

the anecdote

Many years ago I was involved in an ERP implementation in the public mental health sector and remember some simplification decisions during the program.  After go-live unit managers were complaining very loudly about having to click a couple of extra buttons on a screen as part of a common process in a high dependency unit; we sat in our office, which was an old operating theater, and worked out it took about an extra 20 seconds, which they needed to do a few dozen times a shift.  Clearly this was not a big deal and not worth customizing the package which would add complexity, impacting upgrades.

I decided to go out to this unit to talk to them and see what was actually going on.  This changed my viewpoint forever.

The little office they had the computer in was in the middle of this high dependency mental health unit, with big glass windows and two corridor openings.  There were highly animated people everywhere and it was a very confusing and unpredictable environment; I really did feel exposed and uncomfortable.  For those staff that were working, often understaffed, taking your eyes off the situation for even a few seconds exposed patients and staff to potential, and very real, risk.  We thought we had removed complexity, but we had simply moved complexity, and moved it to a very dangerous place.

I went back to the ex-theater office, sucked back some oxygen, and registered the ERP module for customization.

adult-bald-beard-1194196

“Go see, ask why, show respect.” – Fujio Cho, honorary chairman of Toyota Motor Corporation, referring to going to the Gemba (the real place of work)

complex vs complicated

Some systems theorists believe that we confuse complex situations with complicated situations, and the difference matters a whole lot in enterprise architecture.  An iPhone for example is complicated having many different components and extensive software, but this is all hidden by the user centric design so for a user it is generally not complex; this is because most people find iPhones highly intuitive, predictable, and easy to use.  Personally I don’t so I use an Android!

Complicated problems are still deterministic, and therefore can be solved logically in a reasonably accessible way, whereas complex solutions tend to be more multi-factored, interrelated, with many unknowns, making outcomes harder to predict.  If you look at managing complicated software engineering problems then typically a well structured logical approach will solve it, and once solved they tend to be open to automation techniques.

A complex engineering problem however is not easily “worked out” using logical thinking and whiteboards; even sticky notes can struggle here.  Leading approaches use things like lateral thinking, systems thinking, HCD, design thinking etc.  Typically you come up with a hypothesis and then experiment to find out if it works, adapting to feedback, and pivoting.  It is easier to observe actual behavior, than try to predict expected behavior accurately.

“Complex situations do not lend themselves to a solution, and it is folly to spend the time, energy, or effort even to attempt to create solutions. Yet this is exactly how the complicated way of thinking works” – Dr. Rick Nason

So we need to look beyond simply how complicated something is when determining complexity.  A great example of a complicated solution is a modern passenger aircraft.  An Airbus A380 is made up of about four million individual parts produced by 1,500 companies from 30 countries around the world.  Their software executes over 100 million lines of code.  These awesome machines are most certainly complicated, but if they were complex we wouldn’t travel in them.  Generally they are highly predictable given the pilots training and industry processes, and therefore are safe.

a380, air, airbus

In comparison a rugby ball has significantly less than 4 million parts.  Basically an inner bladder and an outer synthetic layer.  However ask any rugby player and they’ll tell you that trying to rely on the bounce of the ball is highly risky; it can bounce in any direction.  Technically it must be deterministic, but given the number of variables and decision making time you cannot reliably predict it.  Effectively in the context of the game, this is complex.  This is similar to the story of the button in the metal health unit, where the complexity is in no small part due to the context of usage.

USO-Sale_Sharks_-_20131205_-_Ballon_flottant

Photo by Clément Bucco-Lechat, acquired from Wikimedia Commons

You may or may not agree with this definition of complex verses complicated, but to avoid an etymological debate lets just agree that complexity is nuanced and an understanding of the full context is necessary to understand if something is complex in practice.

We care about distinguishing the complicated from the complex because we don’t want to remove complication that can be handled logically, and end up creating complexity elsewhere in the wider system (people, process, and technology) that cannot.

On complexity – “Think ‘manage, not solve'” – Dr. Rick Nason

solving the complicated

It is probably complicated if it:

  • Is intricate with many parts, components, modules, lines of code, actors etc
  • Reasonably predictable behavior in context

Approach with solving techniques like:

  • Logical determination, such as decision trees
  • Structured testing approaches
  • Documented execution processes
  • Test automation

Things to avoid include:

  • Treating it as complexity that must be removed, where attempting to remove it may actually create real complexity elsewhere, such as removing the Ground Proximity Warning System (GPWS) from an A380 aircraft to reduce the amount of code
  • Allowing the complicated to bloat and morph into complexity over time as many interrelated complicated systems almost certainly become complex

managing the complex

It is probably complex if it:

  • Is hard to predict its behavior in context
    Note: often it will be intricate as well, but it may not be (rugby ball)

Approach with management techniques like:

  • Defining hypothesis, testing, observing, and pivoting
  • Advanced analytics may provide benefit
  • Carefully determine if the complexity is buffering people and processes from additional complexity, particularly where that might add operational cost or risk to your businesses, or more importantly expose customers to pain

Things to avoid include:

  • Trying to solve complexity with logic alone
  • Pushing the problem to others with assumptions and risk acceptance only
  • If you decide you ought to remove the complexity ensure it is truly being removed from the organisation and not just move to another technical domain, or worse, onto the business and/or partners and/or customers

a note on systems thinking

It is beyond the scope of this post to unpack systems thinking, but this is a great area to explore to gain a more in-depth understanding of how things interrelate to cause challenges.  Thought leaders include Russell L. AckoffLudwig von Bertalanffy, or for a more contemporary view Donella Meadows book, Thinking in Systems, is an excellent read.

Image result for thinking in systems

 

3 Comments Add yours

  1. Nice blog. Your description of complex reminds me of wicked problems.

    Like

    1. Thanks Nick. I had not heard of “wicked problems”, but that is an interesting concept. I guess my reference to the A380 is that you can actually solve most problems you’ll come across, but it just might not be worth it in the context. Of course what architects tend to do with wicked, or perhaps naughty problems, is to make a whole heap of improbable assumptions and then say it can be solved.

      Like

    2. John Rogers says:

      Hi Nick, thanks for pointing out ‘wicked’. I believe ‘complex’ is where engineering and quality processes work well. ‘Complexity’ is where they do not give a useful solution. And ‘wicked’ is where they make the problem worse. Peter Senge’s 11 laws tell us the symptoms of problem spaces where engineering and quality logic is breaking down. However, there is a process and logic that works for all the problem types. It stems from John Boyd’s Observe, Orient, Decide and Act (OODA) process loop, which is the core of modern military, strategic, start-up, and sports thinking. Systems thinking, engineering and quality belong as important mental models in the Orient step to filter the information from the observe step. The important OODA lesson is: “when you have a mental model that has a mismatch with reality (aka complexity) it is time to improve or replace it”. Cheers.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s