Monthly Archives: August 2013

Clinical Safety and sharing data

An institution has a health record eco-system that is distributed and poorly connected. Due to technical, procedural and policy issues, the data is divided into a series of different silos, and there’s not a lot of inter-connection between them. Though – presumably – the systems have connection points through the patient clinical process, because of differences in perspective and purpose, different information is collected, and because of various system design approaches and various lapses in system continuity (the fog of war), the data is a little out of sync.

This is hardly unusual in healthcare.

Then, you create a new integration point between two of the silos, and present newly available information at the point of care. Mostly, this is good, because the clinical decision makers have more information on which to base their decision. It’s more likely that they’ll pick up some fact that might otherwise have been ignored.

Only there’s a problem: the new information is occasionally wrong. The clinical users aren’t fully aware of the various error vectors in the new data. Processing the new data takes extra time. And then the refrain starts – This is Clinically Unsafe!!! It Must Stop!!!

This is regular problem in integration – it’s almost guaranteed to happen when you connect up a new silo. Here’s yet another example of exactly this: “PCEHR EXPOSES MORE EXAMPLES OF PBS ERRORS“, and, of course, most people will report this as a problem for the pcEHR.

Only, I’m not so sure. The problem is in the underlying data, and it really exists. The PCEHR just exposes the problems. It’s no longer hidden, and so now it will start being dealt with. Isn’t that the point?

Generically, exposing a new data silo will cause these issues to become visible. How do you decide whether this is a less or more clinically safe?

  • To what degree is the information wrong?
  • How much would the clinical decision makers rely on it in principle?
  • How much would the clinical decision makers understand the systemic issues with it?
  • How is the underlying purpose of the data affected by the errors that were previously undetected?
  • What remediation is available when errors are detected – can anything be done to fix them, or to reduce their occurence?

On the subject of less or more clinically safe, many people do not understand one aspect of the clinical safety systems approach. Here’s an illustration:

safety-process

 

You change a clinical/workflow practice. The change eliminates a set of adverse events, doesn’t eliminate others, and introduces some new ones (less than it solves, hopefully). Ideally, the numbers of the latter two approach zero, but in practice, if the number of the last (new events) is lower than the number of the first (eliminated events), then the change is worth making anyway- probably. There’s two problems with that:

  • The new events are usually *worse* – more systemic, at the least – then the old ones. That makes for better stories, particularly in the news
  • Distribution of blame to the maker of the change is unequal (introducing new errors is more costly than not eliminating existing errors)

The debate over data quality issues in the pcEHR – and in other contexts – will continue, but it most likely won’t be informed by any insight around how to think about clinical systems change and safety.

But for integration experts: The list I provided above is a useful place to start for issues to consider when doing a clinical safety check of a new integration (cause you do do that, don’t you!). However many real data quality issues are only visible after the integration has been put in place – they don’t even arise during the testing phase. So you need to put systems in place to allow users to report them, at least.

Question: Starting with HL7

Question:

I have developed a application on MVC 4 Platform with sql 2012 database. Now I want to make the application hl7 version 2.7 compliant.
Can u please let me know steps that i need to follow to achieve HL7 compliance.
It is my first project on HL7 and I don’t understand how i manage my database for HL7.
Please if u can write steps that i need to follow to achieve compliance..

Answer:

Well, HL7 is an interface standard, for exchanging content. So you don’t just “achieve compliance” by doing something in your database – you use HL7 when you exchange content between your system and another system. The first place to start is when a customer or user of the system wants some data exchanged with another system – then you can start thinking about HL7 compliance:

  • Characterise the data exchange (what data is exchanged? Who initiates the exchange? How are object identified? what codes are involved?)
  • Agree how the data is represented in HL7 (what message types, what elements are used)
  • Get test data
  • Develop the interface
  • Test using the test data with the other system
  • (Optional) use a validation service to provide evidence of conformance (but this mightn’t be the same as proving it works)

An alternative, more produce focused approach is to provide support for a kind of exchange in your product, but this involves much deeper knowledge, and I don’t think you can or should try for this without first having done several one-off interfaces so that you know what variability your tooling has to support.

Aside: I know that a lot of people think that this variability is a weakness of HL7, and some parts of it are. But hyper-variability is a feature of the environment in which HL7 is used, and so variability is a feature of the standard.

Note that interfacing using HL7 will create implicit requirements for your database, but these are highly dependent on your architecture and requirements, so there’s no general documentation about what these requirements are.

 

Clinical Safety and Data Duplication

Here’s how to kill patients:

<manufacturedProduct>
  <manufacturedMaterial>
    <code code="4683L" codeSystem="1.2.36.1.2001.1005.22" 
      codeSystemName="Australian PBS Code" 
      displayName="Kaltostat (168212) 7.5 cm x 12 cm dressing medicated" />
    <name>Atenolol</name>
  </manufacturedMaterial>
  <manufacturerOrganization>
    <id root="1.2.36.1.2001.1005.23" extension="CC" 
       assigningAuthorityName="Australian PBS Manufacturer Code" />
  </manufacturerOrganization>
</manufacturedProduct>

This is a fragment of a presription taken from a real CDA document (not in a production system, though).

The problem here is that the code and the name identify different medications. Presumably, the prescriber intended one, and the software supplied the other in error.

Now in this case, it’s most likely that anyone handling this prescription will notice. There’s 2 cases:

  • The prescriber intended to prescribe an anti-hypertensive, but the software provided the wrong code, and the receiver sees to apply a dressing instead
  • The prescriber intended to specify a dressing, but the receiver sees to administer an anti-hypertensive instead.

Now these are sufficiently different that the dispenser or adminsterer will almost certainly notice. Especially when the instructions say “Apply to wound as directed, replace when strike through occurs initially daily but then no longer than every 7 days until wound has healed.”

But what if the 2 medications weren’t that different in terms of form or adminstration method, but still very different in terms of metabolic effect? Quite likely, the wrong medication would be prescribed.

The error in this case is that the <name>Atenolol</name> has wrongly appeared in the data – it’s a carry over from the previous document where “Atenolol” was the correct value. This is a software bug. The downstream consequences are quite unpredictable, since the information is provided both wrongly and rightly. It depends what the consumer of the information is shown.

Aside: Carry-over like this is the source of the most dangerous clinical safety issues I’ve seen, including in production systems. How does carry-over like this occur?

  • Using global variables (or writing temporary files to fixed names)
  • Not clearing out memory ruthlessly or initialising objects/records properly (including compiler bugs)
  • re-using an document object model for multiple cases instead of recreating it
  • not understanding the difference between mutable and immutable objects (or failing to protect against the side effects of this problem – or sharing immutables between object models when you think you’ve protected enough)
  • Allowing threads to exist at all (or, since there’s not a lot of choice, failing to 100% get your thread contention management right. 99.999% is not good enough – a typical clinical system does 1000000+ transactions a week, so my rule of thumb is that one in a million is once every week)
  • Wrong test conditions in population code (possibly combined with the above things as well)

Irrespective of those things above, this can only occur when there’s a failure in process as well – any vendor selling clinical software needs defense in depth against these kinds of errors, and many others as well. Testing, in particular, but it’s very hard to test concurrency based issues at all – prevention is way better than detection with these. More than anything else, developers need to be empowered and encouraged to put everything else aside and chase clinical safety bugs anytime they get a whiff of them. These bugs kill people. In the worst case, wrong data is associated with a patient, and the patient clinical situation is equivocal, and the wrong data is decisive, and the consequent wrong treatment kills the patient. I’ve seen this.

Note that it’s also possible that this is a user transposition or copy/paste error. I’ve seen lots of them too, though this case isn’t one. Computers don’t make those by mistake very often, but when they do…. it’s really pernicious.

But what I really wanted to talk about with this case is the issue of duplicated data.

Clinical Safety and Duplicated Data

Duplicated data is common in HL7 formats. Narrative /data in CDA. The same in FHIR. The coding data types in v2, v3/CDA, and FHIR). Code vs name in various RIM classes and v2 segments. Names vs identifiers everywhere. And this isn’t an HL7 practice either. DICOM and IHE do it – really, at this level, DICOM, HL7, and IHE are all one community. The standard themselves – v2, v3, CDA, FHIR, Dicom, etc – all come with human readable and computer processible formats.

But outside this it’s common practice too – and html page with embedded links or micro-data (first is ubiqutious, the second is less common). Word Documents with properties – it’s a ubiquitous thing in IT, because we have two consumers with radically different capabilities: computers and humans.

However, we still regularly hear that duplication is a clinical safety issue (see here for an older example), because in cases such as these, there’s an additional risk introduced by the difficulty of keeping the two forms in sync, and the problem of different systems using different parts of the output and getting different outcomes.

Yes, that’s a problem.

But how much of a problem? I think it’s rather fractional. It’s not that I think that it’s easy to keep different things in synchronisation with each other – that might the hardest problem of all in IT. It’s more than keeping things synchronised is just as much a problem in a linear process (transforming from your database to the exchange format), and multiplying the independent information paths through the path (2 fields derived from a single fact earlier in the process) simply means that your problems are more likely to detectably leak as well. And given that detection/characterisation is the hardest part of production bug hunting, that’s not a bad thing.

For instance, if the user picks a code in the UI, it’s the developers challenge to correctly reflect that code in a v2 message. That involves a sequence of steps – typically, I’d expect 20 or so steps to take if I was adding a new look up code to a UI in order to put it in a message. At some stage, for any of these formats, I’m going to have to present both the code, and some description text for it. Once I do that, I need to maintain these in parallel as I move through the process. It’s possible that the code/text split will be in the UI, in which case the synchronicity will be the users problem, but usually, the split is performed in the code.

This example is a case in point with regard to detection – there’s a logic error there somewhere. It almost certainly would’ve been a problem in production if it had got that far, but the increased visibility meant that it was detected early. Btw, it’s not my job to find this particular bug, and I can’t describe further, but I think this one has a genuine underlying error.

So duplication leads to an increase in bugs, but it makes bugs much easier to detect and characterise. So I think it’s about a neutral outcome for clinical safety. That’s good, since the desire to do it is driven by solid clinical usability desires.

Btw, this does suggest that there’s an opportunity to develop clinical safety surveillance programs that detect bugs by cross comparing code and text in exchange streams, though think that there’s some pretty big challenges there to get the signal/noise ratio high enough to be useful.

Question: Medication Checking in a FHIR eco-system

Question:

I am MS student and I am doing my thesis in FHIR.
My task is to make current system reactive that is it should generate alert if two drugs can cause reaction, when taken together.

Can you please guide me that in this problem where FHIR will be used.

Answer:
This is actually a missing piece in FHIR at the moment. If you look here, you’ll see a hole for it:

Decision Support 2.5.3

Still to be done.

On the other hand, the framework is laid down, though there’s still some missing pieces.

Firstly, let’s start with the context. I’m going to assume that you have an EHR System (EHR) that is maintaining a patients records, including managing their medications, and you have an expert system (XPERT) that is able to provide medication interaction knowedge.

The EHR system maintain patient records. Firstly, with regard to the patient, there’s the question of what medications they are on. This turns out to be a pretty complicated thing to figure out. There’s 4 relevent kinds of resource in FHIR:

  • MedicationPrescription – the medications a patient has been instructed to take. Or at least, to get their hands on in case they decide they need to take it
  • MedicationDispense – a record from a pharmacy that they actually gave the medication to the patient (or the agent or whatever)

Neither of those means the patient is or has taken the medication at all

  • MedicationAdministration – a record from a healthcare provider that they actually injected the medication, or watched the patient ingest it (and not throw it up?) or rubbed it on or whatever
  • MedicationStatement – a statement that the patient is actually ‘taking’ a medication – or did take it – or feels as though they should claim that they take it, or something.

Tracking a patient’s actual medictions is hard. But let’s assume that between these sets of resources, and an active system with enthusiastic data entry people, we can get a good grip on what the patient is believed to be taking.

In addition to these medication resources, there’s some other resources that might be of interest to the decision:

  • DiagnosticReport / Observations – to get a handle on the patient’s GFR rates for calculating medication kinetics
  • CarePlan – a statement of the intended goals for this patient (perhaps the drug interaction is known and tolerated? – but this is advanced usage)

So the EHR system needs to share these with the XPERT system, and then ask the XPERT system for an opinion, and the XPERT system needs to return it’s opinion to the EHR.

Getting a decision

At this point, the EHR needs to query the XPERT system. There’s many ways that this can be done. Here’s some choices:

  • The EHR can bundle up everything it has on the patient (or everything that might be relevant) and submit it to the xpert system with a query asking for the decision
  • expose the patient record to the xpert system (using the FHIR API), and then post a query to the xpert system
  • expose the patient record to the xpert system (using the FHIR API), post the query in the same place, and wait for the xpert system to see it (by subscribing to changes)

Then the EHR system gets the response back.

Query/Response

Now – what’s the query, exactly, and what’s the response?

That’s work that hasn’t been done yet, I’m afraid. But HL7 is actively working on these kind of decisions, so I expect we’ll see this clarified fairly soon. If it’s your thesis subject…. you need to get involved in HL7 (pharmacy work group) and push it along, perhaps by providing a sample implementation for people to evaluate.

 

Technical Error in CDA Implementation Guides for ETP, PCEHR

There’s a technical error that runs through most of the CDA implementation guides that we have, including ETP, and other pcEHR related ones.

Background

The problem relates to the codes used for HealthcareFacility.code:

cda-facility-role

 

Here, ServiceDeliveryLocationRoleType is a value set which contains codes taken from the RoleCode code system. The difference between a value set and a code system seems to be easily confuse people.

A code system defines a set of codes and gives them an assigned meaning. Sometimes, it creates relationships between them that provide additional information about how to use the codes and/or understand their meaning.

A value set is a list of codes defined by other code systems. A value set doesn’t define it’s own codes, doesn’t create it’s own meaning. Value sets are used to define a set of codes that can be used in some context. (Aside: “Value set” is a terrible name for this idea – so utterly misleading. I very much wish we could change it).

Part of the confusion is that very often people leave the value set out of the picture – they say, “use this code system for this element” (i.e. LOINC for test result name). But what this actually means is “use the (set of all codes) defined by [the code system]” (i.e. use (all) the [LOINC codes] for test result name)) – where the value set is “(Set of all codes)”. Where this is useful is because most of the time what actually happens is that you need to say “use (this subset) of the codes defined [by the code system]. Sticking with LOINC, that’s because LOINC defines all sorts of codes, not just codes for test result names. So there’s always a value set in the picture, but sometimes it’s implicit.

So the diagram above is saying:

“Use the (set of codes that represent service delivery location types) from the [RoleCode code system]”

When this is represented in the CDA document, we do this:

<code code=”HOSP” codeSystem=”2.16.840.1.113883.5.111”
  codeSystemName=”RoleCode” displayName=”Hospital”/>

Here, the code “HOSP” comes from the code system “2.16.840.1.113883.5.111”, which is RoleCode. There’s no need to represent the value set “ServiceDeliveryLocationRoleType” because that has nothing to do with the meaning of the code – the meaning of “HOSP” is defined by the RoleCode system, and so that’s what we represent.

Advanced note for people who like hard stuff: it’s not quite 100% true to say that the value set has nothing to do with the meaning. See here and note that a later version of the data types (not usable in CDA) caters for also representing the value set in order to support advanced edge case processing.

The error

Unfortunately the CDA Implementation guides commonly have an error, which manifests in ETP (the released Australian standards), and in several PCEHR specifications, notably the NPDR Prescribe and Dispense documents, and the discharge summary specification. The error is that the OID assigned to the value set HL7ServiceDeliveryLocationRoleType  has been used instead of the OID assigned to the RoleCode code system, so that the code looks like this in the CDA document:

<code code=”HOSP” codeSystem=”2.16.840.1.113883.1.11.17660” 
  codeSystemName=” HL7ServiceDeliveryLocationRoleType” displayName=”Hospital”/>

In effect, this code claims to be something different to what it is: “HOSP” as defined by the RoleCode code system.

This has the potential to cause a great deal of confusion: there’s no real difference in meaning, but computationally it’s easy to think that these are different by mistake. Particularly given that the OIDs are so opaque and share such a lot of common root. Testimony to how easy they are to confuse is the fact that this error has survived multiple repeated reviews at many stages, including by me, and has already been implemented by many implementers.

Worse, some CDA IGs are using the correct code system “2.16.840.1.113883.5.111” for the same codes.

The solution

Given that we now have CDA documents in production that use both the wrong and right codes, the only solution available to us is to not note that for the purposes of CDA usage in Australia, codes with the OID
2.16.840.1.113883.1.11.17660 are the same as codes with the OID 2.16.840.1.113883.5.111 when those codes are in the ServiceDeliveryLocationRoleType value set. Formal advice will be issued through NEHTA channels for the PCEHR documents in due course. This blog is just a heads up in the hope that it helps implementers not get caught out later.

 

Webinar: Embedding clinical safety in medical software implementations

Tomorrow I’m giving a webinar for MSIA members called “Embedding clinical safety in medical software implementations”. MSIA members can consult their MSIA releases for webinar details.

In this webinar, I’ll be looking at clinical safety from a vendor/system developer point of view

I’m going to be focusing on the real world challenges faced by people who make decisions about how systems will work. Usually this is vendor architects and analysts, but can also include developers, in-house teams, and government architecture and purchase teams.

Clinical safety is a real challenge for vendors. Here’s some reasons why:

  • Most clinical safety advice focuses on point of care, but systems are only partial solutions, so advice is rarely directly applicable to system designers
  • Any designer will get a diverse and conflicting array of opinions about what clinical safety means. Approximately 1 per clinician (though it can be more!)
  • Clinical information is more nebulous and the gap between the defined structures and clinical operations is wider than in other industries (as far as I can see)
  • The developer/designer knowledge gap is very wide
  • Healthcare is about, well, health, and the lack of it. Mistakes have much more significant adverse consequences than in many other industries. Also, healthcare is about relative risks, not absolute ones – risky things happen all the time
  • There’s a value perception gap around safety – clinical users think that vendors don’t care, and vendors are sure that users won’t pay anything extra for safety. And vendors are almost all backs-to-the-wall with regard to basic profitability right now
  • Most of the systems have a slower turnover time in basic design philosophy (measured in decades) than clinical safety thinking, which is advancing quickly
  • The loop from error detection to source designer is completely broken

There’s more, but that’ll do for now.

The webinar will review the space, the challenges we face, what MSIA  is doing about it: the MSIA Clinical Safety committee. Also, I’ll be introducing a workshop I’m going to run for MSIA members in November, on behalf of the committee, where we’ll focus  on practical issues around clinical safety in product design, with a focus on case reports (where they are available and repeatable).

p.s. Note that in this work, I’m not really going to focus on programming/coding safety, though I will mention a few hair curling stories of how things can go wrong, but I think there’s more than enough expert advice on this subject.

 

FHIR DSTU / Connectathon Stable Version Posted

We’ve posted the stable version of the FHIR specification that will be used for the DSTU ballot opening in a few days, and also this version will be the one used for the connectathons in Cambridge and Sydney. Some minor cosmetic changes are anticipated, but nothing of substance, in particular there will be nothing that changes the schema anymore. So we are now in the connectathon freeze.

The new FHIR spec has undergone a major cosmetic redesign. The old sidebar was very functional – at least to me – but also attracted a lot of attention about being unfashionable etc. And then it got too long, and it wasn’t good anymore. So with the help of Studio-Joyo (thanks heaps), we’ve updated to a very current look (based on bootstrap, for those who care). We’re still settling things down, and I think it looks much much better, and we’ve got it working as efficiently as the old one too after a little bit of teething (though I still have to figure out what’s happened to the Google search button). It’s been an interesting process – the requirements for a web site and a formal standard aren’t quite in alignment. But we’re nearly there after a marathon effort.

One thing of note about this – prior to the DSTU, we underwent a QA process, where a bunch of volunteers worked through the spec looking for misspellings, broken links, logical fallacies, and other obvious mistakes. I thank those volunteers very much – the specification is much improved because of their work. Unfortunately we couldn’t fix everything, but anything we didn’t fix will be carried forward to the ballot.

Servers

In addition, I have updated my server (http://hl7connect.healthintersections.com.au/svc/fhir) to the same version as the posted specification. The other test servers will catch up as the connectathon approaches. I can update quicker than most, because I run with less manually written code than all the rest of the servers, so I can update quicker – the only manually written code I write is the actual indexing routines. I’ve considered trying to automate them too – the xpaths are published for most of them – but I’ve found that this a good way to perform my own QA check on the resources (and also I run closest to the editorial cycle)

 

p.s. Some people have complained to me that the FHIR site on HL7 frequently fails to load. This happens to me all the time – the HL7 DNS is a problem for me, and the first request fails when the address is not cached. I thought it was just me (or telstra here in Australia, but I’ve had reports of the same problem elsewhere. If you have problems accessing the HL7 web site, drop me an email

Challenges funding standards development

My good friend Dave Shaver writes:

The key is that the benefits of HL7 are often enjoyed by parts of the food chain that do not directly fund its development

Read the whole post – worth your time if you are remotely interested in standards development in healthcare.

I think that what makes something a standard is the very fact that benefits are accrued somewhere else than where the cost is incurred. What this means to me is that all models of standard development are financially broken. The common refrain that you often hear around HL7: “All models are wrong, but some are useful” (George Box) is even more applicable to HL7 itself:

Any funding model is wrong.

But some are useful – at least one, we can hope.

However I remain concerned that HL7 have changed their funding model without fundamentally changing what they are as an organisation – the two things were matched together, and now we have dissonance. Following Kuhn, then, that would mean that a pradigm shift is in the way. Well, we can only hope. Because right now, too many members are hung up on irrelevant internal disputes about how control and resources are divided up (Affiliates, US affiliate…).

This whole question will soon become irrelevant if the organisation doesn’t fundamentally re-assess it’s whole purpose and structure. Some in HL7 – including in the leadership team – understand that this has to happen, and are working towards it. But too many insiders (volunteers, I’m looking at us) are still stuck on the first point – dividing up the spoils. But that’s a waste of time if the size of the spoils starts shrinking rapidly, as it might very well do in the future if a paradigm shift doesn’t come.