HL7 needs a fresh look because V3 has failed

A few months ago, I posted a call for input into the HL7 Fresh Look Taskforce. HL7 wouldn’t need to have a fresh look task force if it hadn’t lost it’s way, and it wouldn’t have lost it’s way if v3 hadn’t failed. But it has:

HL7 v3 has failed.

Now a few readers are going to stop reading at this point and rush off and go quoting me saying “v3 is broken” or “you shouldn’t use v3” or “national program XXX shouldn’t have used v3”, but I haven’t said any of those things – v3 can be made to work if you provide enough skills and resources, and for some things, it’s the best solution (CDA, for instance). But overall, v3 has failed to achieve the goals that HL7 has (being the general best solution to everything), and is not a vehicle that can take the organization forward from here.

That’s a sad and painful admission for an organization that has invested *so* much work into an idea that seems to have so much promise. And HL7 really has worked hard on v3. But where we’ve ended up hasn’t been where we expected to end up.

I see it as if, back in the mid nineties, we stood on top of the mountain called v2, and said, “from here, we can see a great mountain over there, let’s climb that one.” And v3 was a monumental climb, much bigger than we expected (and I have a huge amount of respect for the people who pursued the climb with an awesome vigor and determination, most especially Woody Beeler). But we’re at the top now, standing on top of the mountain looking around. And to be honest, the view isn’t what we thought it was going to be.

Why?

  • the V3 design process imposes consistency on our standards (it’s primary goal) but the inconsistency is mostly not of HL7’s making (example)
  • the V3 design process is technology and platform agnostic – but that just makes implementation more costly
  • the V3 design process depends on design by constraint – see my previous comments on this
  • the v3 process really required all the possible modeling to be done up front – the price of change is too scary – and therefore produced models that weren’t implementable directly (too much! too big!) without support from a large program and highly skilled insiders
  • the V3 models (RIM) created semantic interoperability but not clinical interoperability, and it was only once people tried v3 that we discovered what the difference was
  • we tried to satisfy too many different implementer interests, and ended up satisfying none of them (see my comments on context of interoperability, the HL7 community, the xml consensus, and lack of real compromise)
  • we just never got anything compelling in the space of specifying the behavioural aspects of interoperability

Now I’m not saying that you can’t make v3 work – in fact you can, quite well indeed, if you buy into the “sandbox”. If you’re prepared to adopt the entire specification stack and invest in making it work, then v3 works quite well. But the walls to the sandbox are quite high, and v3 is not an opt-in project where that makes sense (unlike openEHR). It’s a general interoperability standard, to be implemented in lots of ways by lots of different people, mostly by other people’s choice.

I’ve worked hard, with others, to try and lower the walls over the last few years. But we’ve by and large failed. Changing a specification is very difficult. The people inside the sandbox don’t like us diluting the purity of the original ideas, and for most users, the outcome hasn’t been much different – it was too little, too late.

It’s not that there hasn’t been good outcomes from HL7 v3. CDA is obviously a good outcome. But we have to be realistic – CDA isn’t magically successful because of the parts it gets from the v3 process, but almost in spite of them (I can’t tell you how many people have told me that they’d rather have v2 segments embedded inside an html like document).

So it’s time to look again (that’s what the fresh look taskforce is all about) – what are we doing? Where are we going now? I, for one, don’t think we can stand happily where we are. This is IT, and the only constant is change. So what are we going to do now?

p. s. I have my own ideas – check back tomorrow…

Update: please be to sure to read the counterpoint argument

Update 2: See my own ideas, and I posted a summary of comments

29 Comments

  1. Peter Jordan says:

    I don’t see HL7 v3 as a complete failure, the expectations were just too high. It has given us a good document standard, backed by a fairly solid, generic, object model and a workable set of data types for interoperability.

    Given that the vast majority of E-Health data is still stored in proprietary, relational databases – that are tailored for particular applications – I believe that any attempts to extend HL7 v3, or any other all-encompassing standard, beyond the integration layer will remain on the fringes, for the foreseeable future.

    Other than unrealistic goals, I’d say that the biggest mistake has been the failure to provide tools to ease the path to adoption by vendors.

    Obviously, the v3 messaging hasn’t been a success and I’d be quite happy for HL7 to use an upgraded version of the v2 standard for transporting CDAs – or just leave that part of the jigsaw to the likes of IHE/XDS.

    Apart from tooling, my view is that HL7’s efforts should be directed into improving CDA – for example the linking of text to atomised data and the rendering of text. Whether that can by done with newer Presentation Layer technolgies such as HTML 5, JSON or by the improved use of XML, I don’t know.

    What I do feel is that the worst possible thing that HL7 can do now is to take the big bang approach and attempt another re-design from the ground up. That really would represent the triumph of hope over experience – not to mention the risk that it would destroy a lot of confidence in HL7.

    So, just re-visit the scope and deliverables of the HL7 project and focus on improving the core technologies. HL7 v3 is not the biggest road-block on the path to E-Health interoperability: that is clearly the area of coding Standards – in particular, the lack of fully-coded clinical data in Primary Care databases.

  2. andrea ceiner says:

    It’s a matter of reliability and trust. Let’s go on with V3 (or V4) to make it better. Reduce the objectives and focus on tools, artifacts and services (utilities).
    Mistakes can be corrected. Failure means a final stop, an incredible amount of money and time wasted and no more a reputation to spend. These are not good times to say “let’s throw away money”.
    Keep the good things, abandon some others and go on.
    Bye
    Andrea

  3. HL7 v3 has found its place in the playfield of standards. The v3 standard has not failed at all. It is all about what you do with it. So, if you think that HL7 v3 has failed, where did you go wrong? So, if we think that HL7 v3 has failed, where did we go wrong?
    In The Netherlands we experience great benefits using the v3 standard. Maybe because we invest a lot of extra manpower to use it in the most beneficial way. And that is because we know that no standard can be ‘heaven on earth’.
    Again: it is all about what you do with it, what we do with it.

  4. Thanks Grahame – this message needs to be clearly understood, everywhere including in the inner halls of HL7, and acted on. You have summed it up, and the reasons for it, very clearly.

    In the high costs of healthcare interoperability, V2 was part of the solution. V3 has become part of the problem.

    But we shouldn’t abandon V3 and look for another holy grail. Two thoughts about the way forward:

    (1) If you look to climb a mountain, when you get to the top, you will have left most people behind. What we need is bridges between mountains, i.e clearly understanding the relations between different models. Bridges are first-class objects.(for bridges, read precise mappings)

    (2) One-size-fits-all semantics is bound to be clunky and will fail the market test. We need clean simple models for sub-domains – DSLs for clinicians – which are precisely linked, probably via the clunky universal model. Green CDA technology now makes this possible. The RIM is a reserve currency, which you wouldn’t use for everyday transactions; but your everyday currency should be RIM-convertible.

  5. Graham — I can’t wait to read your next installment.

    I also like the way Robert Worden thinks about the problem.

  6. Michael van der Zel says:

    Were is HL7 v1? HL7 v2 is very well known and implemented. HL7 v3 is “broken”? So HL7 v4 will be heaven 👿
    Same thing with Linux. v2.0 is stable, v2.1 is developer, v2.2 is stable, etc.
    You need inbetween versions to explore and discover.
    Some steps are needed to get us further, so I think we are getting there.
    Rome was not build in 1 day 😉

    • Grahame Grieve says:

      Someone – also Dutch – came up to me and told this to me as a joke at Orlando:
      “Have you heard about HL7?”
      “Only the even versions are implementable”
      haaa very funny.

  7. Tony Julian says:

    Remember that V2.3.1 was supposed to be the last V2 version. “meaningul use” has encapsulated V2.5, which means it will last a long time. As long as those who make the rules dont understand the implications, V3 will stay on the shelf.

  8. Ed Hammond says:

    The world has changed. v3’s difficulties were a consequence of many factors, including politics and misperceptions. So be it. Besides, there is nothing bad about a fresh look. With a rapidly changing world, I would have hoped we would be taking a Fresh Look anyway. We learn and react from experience. I praise HL7 for a Fresh Look.

  9. Ewout Kramer says:

    I must admit I do feel sympathy for Grahame’s criticism of Hl7v3. Many of the points he makes have direct consequences for a group probably least served by the current standard: the software engineers who have to build the systems that compose and interpret the messages.

    I have been there. I have tried. And we sort of succeeded, but it was never really as flexible or open as we would have liked it to be, definitely not to the level the standard tries to achieve. Many, many, ISV’s will just shrug their shoulders and opt for simpler solutions. They choose to implement hl7v3 messaging only for some specific scenario’s their national health institute forces them to. And believe me: these implementations have nothing to do with the semantic interoperability promise we embarked on with v3. Much of the semantic aspects of v3 are excess luggage for these men and women who just want to get their message out. Whenever one of the exchanged messages deviates from the examples given in the national implementation manuals, these systems will fail.

    Getting developers on the bandwagon is far more crucial than it seems. Whatever happens, at any ISV the crucial question will land on the (senior)developers desk: “how much time do you think it is gonna take to implement this? Can we even implement this?”. If he cannot find out during the spare time in his weekend, his answer on monday will be “no”. This is how decisions are made.

    This is also the reason the CCD is reasonably succesful: you could just pick up the ASTM/HL7 CCD implementation manual and afterwards have a feeling of how this could work with your application. It is limited, it is well documented. You as a developer can get an idea of all the possible scenario’s you’ll have to implement. In the end, most of us want a standard that tells us how to do things, not how much freedom we still have to implement it.

    Where does this leave us now? Throw away v3? Start a brand new v4? Absolutely not. The domainmodels are valuable. The RIM is a nice generic base for modelers and implementors alike. The datatypes are very useful. Why not opt for something like a “base” set of much more constrained models that cover *most* of the common cases? They should be small enough to be documentable and detailed enough to be implementable. You should be able to use them independently, but still be part of a bigger framework, so they can be easily integrated. Our current modeling framework is strong enough to do this.

    Just make sure my developer’s fingers will start tingling when I see them.

  10. Kai Heitmann says:

    I agree with one of my predecessor writers, HL7 v3 did not fail but the expectation was very high. Yes, HL7 v3 is difficult to implement. Yes, HL7 v3 is complicated. But this is not really a property or “merit” of HL7 v3. It is because health care is complicated and implementing information systems for health care ready for communication is difficult. Or even sometimes unwanted.
    CDA is the most successful family member of v3. That’s good. Its success is based on a common misunderstanding leading to what I call the “CDA effect”: A single model was easily understandable and v3 messages were found to be too complicated… the scope of CDA was then even bent to achieve communication were CDA isn’t really the right thing for. And later many people realized that “more” is needed in order to get real interoperability. That’s were templates were “invented”… But anyway, CDA is great! And I am proud of having provided a bit to it.

    So, #1: HL7 v3 is successful!

    The success story of CDA shows the v3 messages problem. If there would be a choice of using very generic models or very specific models people tend to choose the easiest way. Understanding a specific health care domain and creating application that supports the (rapidly) changing work flow and taking user needs into account is difficult enough so understanding loosely “labeled container” communication (provided by other standards developers outside HL7) would be just enough. No more time to understand more. But HL7 v3 is much more. Its not only a labeled container collection, is carries semantics. And hughhh, why do v3 if containers (or even comma separated files) can do the same. Simplicity wins? Hmmm, health care isn’t simple! And without going into the v3 direction we’ll never achieve interoperability. Its about finding a common language, far beyond the words “container” and “label”.

    So, #2: HL7 v3 isn’t simple. Health care isn’t either. But it provides a mechanism of common understanding in order to achieve interoperability. And this cannot be bought for one and a half cent.

    What really went wrong? What really is needed in the near future?
    #3: A better guidance to all stakeholders on exact this common understanding
    #4: A better guidance to all stakeholders on how to do first step implementations
    #5: A better marketing for v3 (“CDA” did a good job here 🙂
    #6: To be aware that sometimes the beauty of a standard is very hard to implement
    #7: To realize that maybe the 80% perfection is more than needed.

    I think one cannot say HL7 v3 was a failure or was a success. In some aspects it is excellent and in some other it is somewhat ugly. But it’s the normal way of development and as in real life there is almost always an opportunity for doing things better. I look forward to being a part of that.

  11. Peter Hendler says:

    HL7 RIM has other uses not originally foreseen and not just related to interoperability. All of these arguments are about he main use of RIM, to create packages of information to go from one system to another, each system having it’s own data models. It wasn’t part of the use case for RIM to be a data model for integrating data on the persistence layer as in CDRs. There are some large scale examples of this. Since the RIM is one of the most elegant models (maybe the most) capable of representing data from different domains in health care, it is uniquely suitable for basing persistence layers where multiple domain data is to be all in one schema. The Java SIG demonstrated that years ago, and there are various vendor products based on this. There are two basic approaches, the “any RMIM goes” approach which requires pre parsing of meta data in MIF files, and the “Map it all to one universal RIM based model” approach that has been successful with some large scale CDRs. A lot of the other models are organized containers for lots and lots of single well defined structures. But they lack the simplicity and elegance of the RIM backbone where basically, when all is said and done, an Entity in a Role Participates in an Act related to other Acts. That is very simple, and it allows you to model any health care statements in a similar way. It makes the RIM uniquely suitable for the persistence layer which was not on the original HL7 menu.

  12. Lloyd McKenzie says:

    I’m not sure HL7’s goals for v3 were ever as explicit as “the best general solution for everything”, though I accept that was probably the implicit desire/hope.

    Reality is that “best general solution for everything” is an exceptionally tall order. Even “‘best’ specific solution for something” can be hard to achieve.

    I think v3 is “better” than v2 for some use-cases, not as good for others.

    One of the main challenges with v3 is that it pursues things with a rigour that the market, or at least some portions of the market, doesn’t necessarily want. For those who need to use, analyze and live with data, semantic interoperability is critical. For those who just want to sell systems or do their jobs, it barely makes the radar screen and certainly doesn’t justify any significant expenditure of effort. V3 requires a significant expenditure of effort.

    We could well argue that the market is being silly and that semantic interoperability is as vital as most of us think it is and convince ourselves that the market will eventually come around. If we do that, v3 will continue to be limited to those niches where a large party with sufficient resources understands the importance of semantic interoperability and is willing to pay to make it happen.

    Alternatively, we can put renewed focus on “easy to use” such that the incremental effort to achieve semantic interoperability is greatly reduced. How well we can achieve that balance will determine how well v3 (or v3.5 or v4) does.

  13. Gunther Schadow says:

    I am obviously biased, and so whatever I think doesn’t matter much, i.e., that I think that HL7 v3 has brought forward some awesomely useful things such as the RIM, the data types, semantic interoperability, design by constraint, and it’s wonderfully implementable, which is what we have also shown. The biggest problem is the baggage that came with excessive design by committee where every pathological use case always adds its stuff making everything more and more complicated and dilutes the utility density.

    I find the responses interesting, from a wide range of people we hear: failed or not, lets not throw it away there are some good things worth preserving, and they all cite the RIM (oh the evil RIM that supposedly no one understands??!!) and the data types.

    From that I conclude that the important v3 pieces have been a success, now let’s just make them easier to use. There are many ways to reduce the crap and make the good stuff more central and flexibly useful.

    If you look at HL7 v2, the one thing that’s been most useful is the ORU message. And what is that? Observation and ActRelationship. The v3 rendition of ORU (a plain generic one, not a big R-MIM of stuff) is amazingly simple and with ActRelationships much simpler than even the v2 ORU message. So, if we just give people that (essentially what CDA is doing) v3 has done a fine job.

    If that is not done, I fear, the hidden agendas that run this “Fresh Look” thing will manage to run HL7 into the ground.

  14. Lloyd McKenzie says:

    I think there’s a need to distinguish “Fresh Look” as the formal board initiative which may or may not become derailed and “fresh look” which is something that any group of interested and dedicated HL7ers can do at any time – and which I think is far more useful and essential.

  15. […] Graham Grieve, a major participant in HL7 standards writes HL7 needs a fresh look because V3 has failed. […]

  16. Eliot Muir says:

    2011 is a good time to have a fresh look at HL7. It’s a very healthy thing for the organization to feel comfortable with acknowledging some problems and moving beyond those them.

    I think the guts of what could work is a light, normalized data model, edited by hand and expressed in human readable XML or JSON over restful web services.

    A good place to begin is to start with V2 data and ‘refactor’ it effectively.

  17. CDA on its own is not enough as its stateless and without the messaging its only a part of a solution.

    V2 has messaging and does have documents, although they are not well encapsulated and more difficult to see.

    When V3 was started the direction wrt terminology was less clear and there is a significant overlap between CDA (and V3) and terminology models. This is a problem and I doubt we can ever have a terminology neutral model that does not destroy the value of a real terminology.

    Given that we need messaging and V2 is everywhere, trying to add the good bits of CDA to V2, without conflicting with terminology would be a good place to start a fresh look. Enhancing something that works is surely the only safe path given the failure of the V3 process. It may be that we need to develop terminology specific models to overlay on top of V2 to allow proper semantics to be communicated. This has been the aim of the archetypes in V2 work that has bee done in Australia and other places. It allows rich models in a backward compatible way.

  18. Wes Rishel says:

    Replying to Peter Hendler. I would be interested to know if there are *any* truly large-scale CDRs built on a data model and API that is a direct implementation of the RIM. I know of only one. That effort struggled through many versions with performance problems and has faced concerns from a number of sources with regards to the complexity of the API, which indeed implements a persistence layer over the RIM.

    Every other effort that I know of to build a persistence API directly on the RIM has not been proven at large scale and with a large group and diverse implementers.

    More important, however, is the first sentence in the comment. Peter acknowledges that he is describing a benefit of the RIM unrelated to interoperability. This is a bit like telling a composer that his song helps sailors manually raise anchor. Nice to know but sufficiently off-purpose to be unrelated to a discussion of the success or failure of the RIM or V3 as an interop standard.

  19. Gunther Schadow says:

    The funny thing is, when I joined HL7 and v3 was just about to start, I went around with my reverse-engineered v2 data model and asked: why don’t we just take this v2 model, do some mild re-factoring and use that for v3? Then, however, everything was to be redone from scratch. That cost a lot of time. Still the RIM today is quite v2-ish, if you compare it with the v2 ORU message (HL7’s most successful message).

    The problem was then as it is now, that the fresh look is often too fresh. The IT world suffers under amnesia and so does HL7. With that amnesia, fresh looks often reinvent wheels and too often start out trying the square wheel. So it has been, and so it will be again.

    Meanwhile we are putting in successful RIM based implementations that work well especially in new use case nitches, and hopefully this year I get to migrate a 10-million patient EMR to the RIM database. I will just go into hiatus until HL7 fresh look has found out that wheels should be round.

  20. Anon Anon says:

    Not entirely true, FDA has used the standard for Structured Product Labels (SPL) and by mandate has been successful

  21. Shinji KOBAYASHI says:

    I was involved in the first implementation of the HL-7 related blood transfusion. Indeed, we could not succeeded, and I was quite disappointed HL-7 itself at that time.
    However, XML technology was quite immature when HL-7 ver 3 released. In this decade, a number of XML related standards have been proposed and disappeared. We can distinguish it success or fail, but to incubate XML technology, we need victims, try and errors.
    I think we can do better than before.

  22. Gunther Schadow says:

    So what if FDA has been successful “by mandate”? It’s not like Canada InfoWay or UK NHS aren’t using mandates. And it is not like US Agencies can “mandate away” IT failures. The bigger the project, the more likely they are to fail (and fail bad). Also, you can’t usually attribute these failures to just one choice of technology. E.g., NCI caBIG had been slapped badly recently in an independent report and caBIG was not using HL7 v3. My point was you can use HL7 v3 to do things well and cost effectively. But of course, you still have to do them right. But there are 2 things that money can’t buy: (1) love and (2) doing IT right.

  23. Peter Hendler says:

    Back to Wes Rishel. I was thinking how dbMotion has integrated all of Israel and also many U of Pitt hospitals in western PA. They are a fixed RIM based model. They have modified the RIM slightly by eliminating some Participations etc. It is strongly RIM like. There are two styles of RIM databases. The “I’ll store any RMIM as it is” kind and then the “I’ll map what ever comes in to my one universal RMIM”. dbMotion is the latter. Now the VA Health Information Model is also very RIMish. But I don’t know if they actually use it yet.

  24. a V3 success story :

    The group I work for is since 20 years developing distributing and maintaining hospital informatin systems and EMR/EHR applications. We have computerized hundreds of hospitals …. with our own propietary solutions and different technologies. V2 was largely used by us and when we where at the point to choose (again) the “next genertion” model …. we discovered V3 RIM : a clean universal model for our applications and its database model ! OpenEHR was a good alternative, but empty compared to V3 …. everything to be defined (in terms of domain models) from scratch …. while the RIM was already rich indeed !
    We did it, thanks to the support of the RIMBAA community (Peter, Rene’, Michael, Jason, etc…): our new generation of EMR/EHR applications is based on RIMBAA, and it’s alive in production now in 7 hospitals, in different domains (telemedicine included) …. also domains that where not yet defined in the standard …. but having a model gave us the possibility to extend it in a coherent manner. Different applications, in different domains, all based on the same RIM , API (and evolution of JavaSIG), and DB schema (for Oracle, MySQL and Progress).

    Then we started using the same V3 API and RIM for messages, in place of our existing former V2 messages … and … programmers find V3 much easier than V2 ! To make them really happy we shall define a web service interface for them, just to simplify further the implementation.

    A RIM implementation fits well also with BPM and SOA .

    It was not easy …. V3 datatypes (like BAG) are problematic in some contexts … the initial cultural gap and the technical difficulties are huge, but certainly in the human space of the possible things and if you have also a long term company strategy then it is definetly worthwhile.

    V3 is not popular in my opinion for 2 simple reasons:
    – companies that invested in V2 must return on the investment the most and longest they can (thus they try to avoid to implement V3 messages and frameworks to do the same thing they do with V2)
    – the startup is difficult, takes time and needs a strong commitment

    Ciao –Andrea

  25. Georg Heidenreich says:

    About semantic interoperability:

    1. Electronic messaging has to go beyond the container/label approach that all established standards and systems support today.

    2. The HL7 v3 RIM tried to go the long alley to semantic interoperability – by making one giant step, which did not work for many adopters.

    3. Still the approach of model-based specification AND the approach of putting clinical meaning into such models is the right way.
    One can not say, that the street is wrong because somebody fell when walking on that street.

    I admire the drive of Ed Hammond, who asked for a Fresh Look and as a constructive contribution I want to suggest that HL7 Int.

    a) publishes a simplified approach how users can adopt the v3 RIM in their specific healthcare settings – as today’s world is not advanced/standardized enough do use common domain / interaction models. We have to understand that the world in healthcare is quite fragmented and that it takes a step-by-step approach.

    b) defines a simple and precise ITS plus conformance checking methods/tools

    c) publishes clearer guides to the “mechanism of common understanding in order to achieve interoperability” (K. Heitmann)

    Implementing communication on v3 should be defined as precisely as for v2.x – then there will be many successful v3 projects !

    regards, Georg

  26. […] hat mich auf einen Blogbeitrag aufmerksam gemacht, der HL7 V3 ebenfalls sehr kritisch betrachtet – dabei die Hoffnung aber nicht […]

  27. Jeff Brandt says:

    Since is works and I have heard many say “not well” and they prefer V2. Maybe it is just time to refractor the software. You are correct it is very heavy, one size fits all.

    I work mostly in Mobile and FHIR makes so much sense, soon everything will be mobile and V3 will be deprecated. 😛

  28. […] In e-health, openEHR is not a de jure standard but is arguably becoming a de facto standard. It’s gaining use on the ground, among vendors, national e-health programmes, academia and open source initiatives like HANDI in the UK. It is regularly compared to de jure standards in published standards reviews. Cherished colleagues won’t like me saying this, but the evidence is that the closest ISO standard – 13606 – isn’t. Related ISO standards like the HL7 RIM are also dead for practical purposes. […]

Leave a Reply

Your email address will not be published. Required fields are marked *

question razz sad evil exclaim smile redface biggrin surprised eek confused cool lol mad twisted rolleyes wink idea arrow neutral cry mrgreen

*

%d bloggers like this: