Klaus Veil asked me for a comparison of v3 and my alternative proposal (RFH), to help people understand the differences. This is a list of some of the differences.
- the v3 standard is primarily focused on explaining the standard – the process. And with reason – you have to understand the process, from the RIM down, to be able to implement it well. RFH turns that on it’s head: the focus is on explaining what you have to do to conform to the standard. It’s not that there’s actually less to understand – it’s just a different focus on the order in which you encounter things, and why. (My favorite example of the problem the v3 approach causes is the A_SpatialCoordinate cmet. It contains a coherent description of how the parts map to the rim, but the parts are not coherent – and it has passed ballot)
- the v3 process starts from the RIM – the definitions are in the RIM, and you constrain the RIM down by removing optionality and capability to focus on a particular use case, and the instances that are exchanged are RIM instances. In this way, the RIM stands between the business analysis and the implementation model. RFH turns this on it’s head; the implementation is the business analysis model – if there’s a difference between the way business analysts think about the model and the implementation, this difference is explained directly in the implementation model. The implementation model is also secondarily mapped to the RIM – this is required to ensure correctness, but not for interoperability.
- v3 is exploring various forms of simplified xml. The mantra has become, “interoperability is a transform away from the RIM”. RFH embraces this – interoperability is the focus, and the RIM is a transform away. And compromise is fought out in the implementer space, not the definitional space
- n v3, the RIM provides the formalism by defining the base types. Typing is innately an exclusive process, so it doesn’t make sense to also define the concepts against different models/perspectives at the same time. The consequence of this is interminable debates about the ontology of the RIM itself, and the perspectives it enshrines. RFH pushes the RIM into the ontology space, where the definitions are open, not closed. This enables additional definitional perspectives to be introduced where appropriate. It would also allow different parts/patterns/concerns of the RIM to be teased apart (where now they are bound together by being contained in a “class” with all that entails)
- in v3, we were forced to model the universe in order to avoid subsequent breaking changes. RFH doesn’t know how to solve this problem, but can leverage the fact that so much of it has already been done. Without this preexisting work, RFH would have a problem….
- in v3, the definitions are “abstract” so that they can be technology independent. This is wonderful in theory, but requires custom tooling to map between the models and the reality (especially in conformance) – and hardly anyone actually understands this in the data types, let alone elsewhere. How painful this has been… RFH is focused on XML first – the concrete model that is exchanged. Other definitional forms are secondary. And the exact form is based closely on a recognized best of breed model (highrise API)
- in v3, there are multiple overlapping static models, some of which are sort of incompatible from some perspectives. These overlapping models represent a failure of governance – the cat slipped out of the bag before we understood what was at stake, and we never got on top of it. It’s a major issue for CDA R3, for instance. RFH can’t solve this – instead, it elevates this problem to a central concern by insisting that resources are modular and separate, and we have to govern it from the start.
- in v3, we have still not figured out a workable behavioral model. RFH starts with a restful infrastructure – this meets many requirements out of the box. Where it doesn’t, additional service models can be built on top by either HL7 or other adopters as desired, by reusing the resources. RFH defines an aggregation framework for resources to enable this, and this allows documents such as CDA to be natural outcomes of the methodology.
- V3 requires custom tooling – both to produce the specification, and to adopt the specification. RFH still requires tooling to manage the definitional space, and ensure it is coherent, though more of this tooling arises logically from existing ontology toolkits. RFH doesn’t require custom tooling to adopt the specification, though the definitions are there to be leveraged, and that would require tooling.
- v3 treats extensions as entirely foreign content. RFH, on the other hand, makes extensions part of the base content and insists that they be related to the existing definitional content. Since most extensions are based on content properly defined elsewhere, this is more natural and palatable to adopters
- by pinning implementor concerns down, and being open definitionally (rather than the converse as in v3), RFH should offer many disconnected potential members of the community to re-engage.
These are the differences I had in my head as I set out to write RFH.
Not all of these differences will be to everyone’s liking. In particular, there’s no necessity for all the parts of the wire format to map to the RIM – and the resources I’ve already put in RFH don’t – they explicitly represent identification and management concerns that the RIM has never engaged with. So it may be that RFH can’t interoperate with v3 across a gateway.
Also, not all these differences I had in mind are properly represented in RFH – or it may be that they even not properly achieved. RFH isn’t offered up as a final specification, ready to ballot. It’s an idea, an example of what could be achieved if we approached the standards process from this different direction, and maybe it’s also a usable basis for further work.
Grahame,
many thanks – this is a good start to explaining what RFH means in practical terms …
Klaus
If you’re going to use the RIM for RFH, you need to do a couple of things to the RIM first. One, map it to the emerging w3c standard for provenance. Take a look at the work of prov-xg to get a sense of what’s going on. It won’t be entirely unfamiliar. Two, abstract Act to Event, and 3, separate records of an event (like observation and document) from the actual events.
I’m saying this because I like RFH and where it’s going, but I don’t want it to fall into the same modeling pitfalls that v3 has.
your blog is protected?
Damn you, autocorrect! That was supposed to be http://subluminal.wordpress.com
Act covers more than Events. Orders, definitions, criterion, etc. are not events. As well, all v3 instances are “records” of occurrences. An Observation Event in v3 is a record of what was observed. A SubstanceAdministration Event in v3 is a record of what drug was administered. A SubstanceAdministration Request in v3 is a record of what drug was ordered (though it may be an “actionable” record, just like a paper prescription is a record, but is actionable).
Lloyd, I didn’t find your comments very informative as to why we shouldn’t separate the record from the event.
Jim, I think we only ever deal with records of the events…?
A document isn’t an event. Observations too, aren’t events. Two observations can occur of the same event, the event that is being observed is not the same as the observation itself.
If we’re capturing it electronically it’s always a record. So there’s nothing to separate. At best, there might be a need to distinguish whether an electronic version of an order is “actionable”, but that’s already supported in v3 (we do it in pharmacy), and there’s certainly no reason to have separate classes.
This Fresh Look project is laden with dangers right from the start. You’re being told to look into an “emerging” W3C specification … as if there hadn’t been tons of failed “emerging” specifications that HL7 (v3) has looked “into” over the years. CORBA being one of them.
Add to it the amnesia of people who were initially involved in the v3 project and now blogging to criticise it, but have shaped exactly the trajectory on which HL7 v3 messaging went.
Now Grahame has build his thing, as have others before. There is no shortage of silver-bullet proposals. Concerning also that people who have not produced success either are eating out the guts of HL7 v3 pretending to be able to judge what is good and what isn’t.
Grahame, it’s fine to make an experiment like you did but presenting that as a wholesale alternative?
I can build a RIM based REST object service (in fact that is essentially what my software is) and things are simple. You do not need to throw away most of the parts of HL7 to which even you and Wed ended up paying lip-service in acknowledging in your recent blog posts.
The question has to start with what does the market really need … and that is not what do outspoken “developer evangelists” cry for, but what does the market really need?
Then the next question ought to be how can HL7 respond to the real needs with what it has.
If a REST object service is what the market needs, if small fragments of information is what the market needs as in the PCAST report, then HL7 v3 can easily fill that need. Everything becomes so easy if all you do is provide an object service!
Unfortunately HL7 v3 set out with an entirely different path (and again those who now purport to judge HL7 from the sidelines in their blogs have had an instrumental role setting HL7 v3 on that path.) On the HL7 v3 path we had to develop large RMIMs because messages were supposed to carry a huge amount of information at once which made them vast and complicated and a nightmare for specifying conformance.
Just take that away and do your outline of a simple RESTful system based on the RIM and everything becomes easy. BUT FIRST you have to ask if that is REALLY what the market needs, again, I mean the market of actually installed systems that have data to share and workflow to connect.
#Gunther
What does the market want? How can we know? I think we can say with certainty that the market does not want v3 as it is. We have experimental evidence with regard to that.
I’m not sure what you’re claiming the market wants, beyond saying that I’m not the one to judge. Which is no doubt true.
So why did I present a wholesale alternative? Simple. Because I tried to explain it, and no one got it. Or if they got it, they argued that it wouldn’t work. A picture is worth a thousand words – so I had to make a wholesale alternative, simply so people could see that there is a better way. One thing that has intrigued about the response is that it’s all about REST – but it’s not really. That’s such a small part of my picture.
I have put forward a candidate. It’s intentially real enough to be an alternative, but it’s hardly ready for use. Argument follows. Yours are the most negative comments yet, and they’re hardly substantial, beyond wondering what the market wants. I wonder that too. How do we find out, other than by trying? What is the market? HL7 members? Or non-HL7 members?
btw, what emerging W3C spec are we being told to look into? I’m not aware of it – unless you’re referring to XML.
Grahame, I didn’t mean to say that you couldn’t know what the market needs. I was saying that this is where the fresh look analysis should start and that it is altogether wrong if it doesn’t start there.
As for this emerging W3C spec I was commenting on such comments as this one above: “map [the RIM] to the emerging w3c standard for provenance. Take a look at the work of prov-xg to get a sense of what’s going on.”
If you claim you have a better solution but address a different problem, then you’re comparing apples with oranges. The point is, if you just need a RESTful object service, then the RIM works pretty fine for that. I actually agree with you that a simpler interaction paradigm would be so much easier and HL7 v3 went wrong by trying to do these big conglomerate messages.
But, the crucial issue is how do we know that these simple interactions based on REST services is what the market needs?
There was a reason for HL7 being reluctant of doing anything other than messages, and that was the vendors did not want to expose fine-grained interactions which you could equate to REST services. This was then, and now is now. So what does the market need now? And how will that be decided?
If you don’t answer these questions you just invite the talkers and evangelists to tout their golden hammers or tools or approaches that may or may not solve any problem (while surely creating a bunch of new unforeseen problems). You won’t get any guidance from this, just the HL7 boat without a navigator tossed around by the winds and the see.
Here is how you could answer the question: by actually studying what people have installed now in various parts of the world and what they need to achieve. Then you think how they could achieve that in general … if simple RESTful services could work for many, then you can do this.
When you say “the market does not want v3 as it is” we still need to know what it wants from v3 and what it is being bothered by that could be changed. Interestingly the responses in to your all-out negative stance on HL7 v3 declaring it as a failure were actually quite encouraging, people weren’t bashing the RIM and the data types. And you and Wes had to acknowledge it in your follow up post. But after that, you’re still tacitly ignoring this response and still operate as if the RIM and the data types were the problem.
#Gunther
There’s lots of specs out there. No reason not to learn, though it’s hard to know what there is to learn from a simpler spec.
I didn’t claim to address a different problem. REST does, but my idea starts with REST, rather than finishing there. I’m obviously going to have to repeat this a lot. I don’t think that highly granular REST is good for everything – you have to extend beyond that. So RFH allows for aggregation of the resources and the exchanges to solve other problems – REST is a place to start.
Going back to my other post, I carefully did not bash the RIM and the data types. But I still had plenty negative to say about the whole edifice that is v3, and I don’t think anyone actually took issue with what I actually said, only with the headline conclusion – but no one argued that v3 as it is will take HL7 forward.
And as far as I am concerned, I’m not still operating as though the RIM and datatypes are the problem (though I’ll sign up anywhere you want to say that the structured vocab is a problem). I just think that they’re being misused. The way you personally use them, I think that works. For a general purpose interoperability standard – they’re not fit for purpose, because they lead to the edifice and it’s problems.
The RIM and the Datatypes aren’t the problem. Requiring implementers to see or care about the RIM is definitely a problem. Grahame’s point is that we can achieve semantic interoperability without requiring most implementers to worry about (or even see) the RIM-based model. And we can do that without throwing away the RIM-based semantics.
The key thing that drives Fresh Look is a growing recognition that we can’t keep doing what we’re doing, because the broad market does *not* seem to like it. So Fresh Look is about identifying problems and examining what we could do better. It’s also about identifying what we want to keep from what we’ve done.
The analysis so far is:
1. Most implementers hate the wire format
2. What HL7 international produces is pretty much impossible to implement directly (aside from CDA, which must be templated to implement and SPL which is more FDA project than UV standard).
Grahame’s proposal addresses #1. It does a limited job of addressing #2, but has the potential to do more.
I don’t think anyone’s saying that RFH is a panacea. But arguing to stick with the status quo isn’t going to get the HL7 organization where it needs to be.
Lloyd, I wasn’t saying that v3 should simply stay the course.
Regarding the wire format, we (Mark Tucker, Bob Dolin and I) had one planned that was a lot nicer, a lot more XML-ish. For example, we were going to flatten out Participations, and ActRelationship elements. But at the time when it was proposed people looked at it and said it was too complicated. They wanted something that would be easily traceable from the model, so it was wished and so it was done.
The problem I have with the fiddling about the XML format now (ITS R2 and new XML ITS) is that when the question was timely, people voted for what we got. You can’t always go back on decisions without losing credibility. Why should I implement any v3 stuff today if tomorrow everything gets changed? V2.x was not implemented in a year either, it took a decade too to get the wide adoption that we see. And also, the SPL experience showed that after the wining about some minor XML “uglyness” the implementers accepted and adopted it.
We also saw another movement inside HL7, people going away from the big v3 R-MIMs and want to apply more of a pure RIM approach with “templates”. That doesn’t seem to square with this new idea of decoupling the XML from the RIM.
The problem has been for a long time that v3 taught the RIM too complicated too much like a thing that’s very difficult to understand. It just doesn’t make any sense that a RIM Observation class is any harder to understand than an OBX segment; it doesn’t make sense that the 4 ways to create observation structures in HL7 v2 (OBR-OBX, OBX sub id, OBR parent order, OBR parent result id) are any harder to teach than the working of one ActRelationship class; and it doesn’t make sense that Participation is any harder to understand than the working of the ROL segment in HL7 v3; it doesn’t make sense that an II (as it was initially) is any harder to understand than HL7 v2 EI+HD. There was a big failure of teaching people how easy it can all be. Instead, the organization always made things more complicated (natural design by committee effect) and wasted 80% of its time on arguing about pathological use cases instead of setting a simple base. Every challenge was always answered by adding more stuff into the standard, without ever accepting a status quo that’s good enough. If HL7 v2 had been managed like that, surely the organization would have eroded its implementer basis just the same.
Sadly this Fresh Look approach is just a continuation of the same v3 mentality to give a voice to outside people and allowing them to throw everything out and over every 5 years.
I say it again: when v3 started I asked why we weren’t doing a HL7 v2 reverse engineering approach with some re-factoring. But v3 invited outside people to put their enterprise data models up, completely losing its continuity with v2. When the RIM was finally simple and more like v2 again (the USAM proposal went along with a 112 page HL7 v2 mapping document [http://aurora.regenstrief.org/RIM/harmonization/usamp-ii%20b.pdf]), the organization saw a lot of people join who didn’t have any stake in v2. SGML folks were just caught in time by Bob Dolin getting the point of the RIM and working it into the CDA foundation. Soon we heard the Australians mumble about GEHR and the UK people about ENV 13606 and then when NHS finally joined the HL7 v3 bandwagon they were allowed to change everything over. And people like Ken Rubin went around for the GPRS project messing up all the models (GPRS failed miserably, but Ken is still around telling HL7 what to do, and as always HL7 listens.) Every new outside group was allowed to stir things up and and continuity was lost. ITS R1 -> R2 is just another one of those debacles, people declaring that wire format compatibility doesn’t matter. There was long no point for an implementer to adopt anything than their private XML format because you can’t trust HL7 to stick with its stuff and develop carefully. And so the saga will continue until HL7 is a pointless organization that once upon a time produced something useful.
Continuity with what, exactly? CDA, sure. However, I don’t think there’s any internationally published v3 specification that’s implemented in a wire-format compliant way anywhere. All the successful (and semi-successful) affiliate implementations have crafted their own tightly constrained RMIMs with widely varying alignment with anything HL7 Int’l has published And the number-one push back from implementers is complexity and wire format. Preaching continuity when you’ve got such limited uptake doesn’t make sense. Especially when a major aspect of the product – v3 messaging – has been deemed “dead” in most U.S. circles.
HL7 needs to continue to listen to outsiders. That doesn’t mean we forget the rationale for why we did things in the first place, but we can’t stick with the same thing forever either if it’s not working.
There was no way that any new structure that was object-oriented or reference model based could have remained wire-format compatible with HL7 v2 and not been a laughing-stock (or had anyone who was willing to implement it). So there had to be a break with that syntax.
HL7 made a decision about the wire format for v3. That decision may have been against your recommendation. I was present for (though not terribly invested in) the discussion about what the wire format should be. It was long and contentious. And the market and experience would seem to have declared (fairly strongly) that we got it wrong.
The fact vendors can do it is irrelevant. Vendors can do pretty much anything if you put sufficient money in front of them and regulatory sticks behind them. The question is how much money there has to be and how big the sticks have to be to get them to move. With v3 as it stands, the answer is “very big”. As a result, adoption sucks. We need to reduce the hurdles (real and perceived) so that the size of the cash bundles and/or sticks can be much smaller. (No vendor is going to do anything, no matter how easy, without a stick or incentive of some kind.)
Technology moves. We could still build all our software by hand-coding assembler or running patch cables. Backwards compatibility is useful, but only up to a point. It’s been well over 10 years since v3 started. It makes sense to look and say “is HL7 where we wanted to be when we started this process?”. I think the answer is clearly “no”. If that’s the case, then we need to figure out why and what we need to do differently.
That doesn’t mean throwing everything away and starting over. I’m pretty sure we’ll keep the RIM and the datatypes. We’ll keep CDA or something like it. Those things have worked. We won’t keep application roles. We likely won’t keep the wire format. Those things haven’t worked. Whatever we do, we need to do a better job of providing a migration and interoperability path from what we’ve rolled out so far.
If we do that, we’re showing that we’re not an organization with our head in the sand. We’re responsive. We fix what’s broken. And we don’t abandon those who were early movers. That’s what a healthy organization should do.
Continuity with v2 would have been possible if a careful approach had been taken consisting of (1) reverse-engineering v2 (2) refactoring (3) forward engineering using better technology. That was then, but today we’re making the same mistake over.
You don’t see implementations of CDA? I don’t understand. Anyway, here is how to find out which specifications should be included in “HL7 lite” — put them up for survey & vote. All voters can vote for which specifications they want, but they have to disclose somethings:
(1) do you currently use this specification in a product or system in production?
(2) do you currently develop to this specification in a product or system that is in production?
(3) do you use this specification in any other way? Please explain: _______________
(4) if any of the above apply, in what country or countries is your product or system operating?
(5) if any of the above apply, how many instances of this specification (document / message count, interaction count) is your product or system currently handling (per day)?
(6) if any of the above apply, how many instances of this specification (document / message count, interaction count) has your system processed / archived as of today?
(7) if you consider this specification essential for HL7 to maintain, what are the 3 most important enhancements or corrections that you would like to see in the next revision?
(i) ____________
(ii) ____________
(iii) ____________
(8) if you consider your (part of the) decision to use this specification in your product or system and considering your experience, would you decide again to use this specification? if not, what alternative would you use instead? ______________
(9) do you know specific products or systems or trading partners with which your product or system inter-operates using this specification? if you can name them, please do.
Add some other info about the person and organization. This sort of survey will tell us quickly what is actually in use. Any specification that has at least 2 trading partners in production would be a candidate for continuation.
All published standards and informative documents should be included in the survey, the DAM documents too, and everything under ballot. The survey should be announced as a ballot and held open for a good time, and all members should be reminded frequently to make their opinion known if they have not already replied.
This survey is vital to gather real evidence before releasing any more opinions about failure, death, success or forgettability of any of the HL7 specifications by anyone with a role in HL7. The fresh look people should start with that homework. All that nagging on technology without this evidence leads nowhere. If you don’t find a way to weigh opinions, this fresh look stuff will just be lost in the wind.
I mention the provence working group because one thing I have noticed is that it has a strong correspondence to the RIM, but more generalized and grounded in theory. Since it’s applicable to outside of healthcare, it stands a good chance of being adopted by large tool producers (Oracle is participating, for instance). Personally, I’m working towards making provenance models core to life sciences domain models. If we both do this, there is a good chance that we can meet in the middle.
@Gunther: I see tonnes of implementations of CDA and various implementation guides there-of. Not much else that’s wire compatible with UV ballot content.
The survey you describe would tell us a little bit about how v3 specs are being used by those who pay attention to v3 ballots. (Even with extensive nagging, that tends not to be a terribly large number.) Even if we got all the affiliates to poll their members, we wouldn’t get a complete response.
More importantly, we wouldn’t get all of those who pay no attention to HL7 v3 because it’s not producing anything they feel they can use. We could attempt to survey these folks too, with an appropriately determined marketing effort, but (as with any consumer group) they often have trouble expressing what product they like until they see it, or even know they need it until it exists.
I’m not suggesting that undertaking an in depth examination of our existing (and potential) customer base’s needs is a bad idea, but I don’t think that doing so in a waterfall approach of “survey first, then build” is likely to work terribly well. Drafting some ideas, tossing them around, bouncing them off the implementer community and then refining makes a lot more sense.
One thing we did very badly with v3 was actually going out to the implementer community once we had some draft v3 specs but hadn’t locked down the approach and methodology and saying “what do you think? Can you use this? If not, what do we need to fix?”. The methodology and artifacts ended up being driven by the producers rather than the consumers.
Doing a survey to find out what’s in use will help us in figuring out what we want to continue to support or transition, but that’s something to worry about once we have something to transition to.