Law #2: Complexity: you can move it around, or externalize it, but you can’t make it go away

Healthcare Interoperability projects are complex; they are made up of lots of people and a great number of data items, each with their own complexity, and usually with subtle interactions between the data, the business processes, and the people involved. Generally, the complexity arises from three different sources:

  • The innate complexity of the problem, the real underlying scientific, clinical and business processes involved
  • Diversity in approach and thinking between different instances of the same processes and data items due to different legacy practices, cultures and personalities
  • Human factors (poor communication, poor motivation, failure to understand, fraudulent behaviour)

It can be very difficult to tease these apart; very difficult to determine whether business process variability is justified or not. This is always aggravated by the fact that any gathering of ten scientists, managers, doctors or nurses etc will have at least ten ways of doing something, all of which are the only way. Even if you can properly resolve business variability and somehow correct for human factors, you’re still left with the innate complexity.

You can’t make the innate complexity go away: it has to exist somewhere. However you break up the project, divide the work amongst the project participants, and write the specifications, that complexity has to be somewhere. It’s not going to go away.

Of course, it’s always possible to make things worse, to divide the work up so badly or to describe it so incoherently that you make things much worse than they need to be. In fact, it often seems impossible to avoid this, impossible to do things cleanly: we keep our solutions as simple as we can, but they get complex all by themselves. Since this problem is so ubiquitous, it can be hard to know exactly how simple a solution gets before it’s gotten too simple, before it’s failing to deal with the complexity.

There’s two ways for a solution to be too simple: it just completely fails to account for the complexity of the problem, or it finds some way to externalize the complexity so it becomes Someone Else’s Problem. The first will inevitably turn into a problem during the implementation phase of a project; such projects often turn into death marches as the complexity slowly is dealt with and the successive cycles of reworking for discovered complexity turn the overall project into a permutational explosion of special cases.

The second kind of simple solution – one that externalizes the complexity – can be a very powerful solution indeed. There’s nothing like turning your own problem into someone else’s problem. One of the hallmarks of successful IT solutions is exactly that – that they solve a subset of the problems, the ones that can be solved neatly, and allow other people to extend the system one way or another to solve more complicated problems. On the other hand, this approach can also backfire badly – it all depends on just who the problems have been shifted to, and whether the cost of that is part of the project, or someone else’s project.

The bottom line is that the complexity that does exist must be handled somewhere, by someone. It’s not going to go away. So everyone involved in healthcare interoperability should account for where the complexity has gone throughout a project, and rigorously resist accepting overly simplistic solutions where it’s not clearly understood what happened to the complexity that does exist. Unfortunately this implies that requirements gathering should actually capture the extent of the complexity of a project, and this is most often a total failure – in particular, people seem hardwired to express their requirements as simple principles with some special cases.

 

One Comment

  1. Gunther Schadow says:

    This is of course true, we’ve been telling that to people for a decade: you can squish the complexity around, but it doesn’t go away. However often this insight has been mentioned in the past, what is the conclusion?

    There may be people who can keep track of their endless requirements sheets with combinatorial explosion of special cases. I can’t. So I wonder what is so bad about “people seem hardwired to express their requirements as simple principles with some special cases.” — if simple principles is what people like, why is there a problem? The problem is in my view that people work with too few well thought out principles and they don’t search for simple principles that actually work. Especially committees are prone to do that. Instead of using simple principles they always find ways of making things more complicated.

    From my share of real health care data that I have seen both in health-care operation, research and regulatory systems, the biggest issue has always been data quality. People don’t understand their own data and mess it up. People build systems that do something and they work as long as you avoid looking to deeply or god forbid try to re-engineer and replace them.

    And software, I don’t even want to start talking about this. People address too many data issue with software APIs, the worst is generated code.

    Health care data and interoperability is all a big engineering mess exacerbated by people who don’t know what they are doing and that problem is multiplied by committees, and the more money the customer has to deploy teams the greater is the crap that comes out at the end. The simple sad truth is that 50% of people doing stuff aren’t really good in solving problems (whatever that is car mechanics, attorneys, doctors, or, yes, IT professionals). If you want to build a house you engage one architect, your car fixed, one mechanic, your appendix removed, one surgeon. Design by big committee and big teams will almost always fail.

    What is needed is real simple, powerful principles that work, and good people who have the guts to apply these principles, and who take the audacity to resist pathological use cases and instead keep the data clean. What’s needed is a few people, small teams, that get something real done with a system based on principles and move out from there to conquer new ground. I am doing that with my customers and I think I can show that I can save them lots of money by taking up new requirements fast. I am doing that with powerful principles that we developed in HL7 v3 in the RIM some time ago.

Leave a Reply

Your email address will not be published. Required fields are marked *

question razz sad evil exclaim smile redface biggrin surprised eek confused cool lol mad twisted rolleyes wink idea arrow neutral cry mrgreen

*

%d bloggers like this: