Monthly Archives: December 2012

Database schema for my FHIR server

Several implementers have asked me what database I use for my FHIR server, and how I structure the database.

I use an SQL database – either MSSQL or MySQL (the cloud server uses MSSQL because Microsoft gives me free licenses via BizSpark). Most of the other implementations use something like MongoDB.

This diagram summarizes the schema that I use:



This table has a row for each defined table type – a name and an internal primary key, along with a set of status fields & flags for how the system supports the type:

  • Supported – is the resource even accepted for this server
  • LastId – tracks the highest integer id value (if integer ids are being used)
  • IdGuids – whether to accept UUIDs logical ids proposed by clients
  • IdClient – whether to allow the client to put to a non-existent resource
  • IdServer – whether to allow the client to post to the resource type and ask the server to create a resource
  • cmd[X] – whether the logical operation is supported or not (note, some operations cannot be supported for some resources, such as search on a binary resource)
  • Versionupdates – whether to insist on version aware updates for this resource type

Note that most systems would fix these things globally for all resources, but I’ve found it useful to be able to vary the policy for different resources – it helps during the connectathon testing phase.


A list of known resources with logical ids:

  • ResourceKey – primary internal key for resource (not visible externally)
  • ResourceTypeKey – type key (reference to FhirTypes)
  • Id – logical Id (char(36) as specified in FHIR spec)
  • originalId – an extra field my implementation tracks, which is the id a client first provided. This is echoed back to clients so they can link resource by their original id. This is not required by the FHIR spec
  • MostRecent – a reference to FhirVersions.ResourceVersionKey, the current version of the resource


An actual instance of a resource (this table is insert only):

  • ResourceVersionKey – internal primary key, not visible externally
  • resourceKey – reference to FhirIds.ResourceKey
  • StatedDate – the date the client claimed that this resource was created (UTC)
  • TransactionDate –the actual date the event occurred (UTC)
  • VersionId –the version is assigned when the resource was created
  • Deleted – true if this is actually a deletion of the resource
  • Format –flag for Xml or Json – the resource is stored as it was received
  • Author – from the http layer, who the author was, if known, or just an ip address
  • [Content] –the actual resource as a binary (gzipped)


List of the indexes that for each resource (derived from the search parameters defined in FHIR spec):

  • IndexKey – private internal key
  • ResourceTypeKey – reference to FhirTypes.ResourceTypeKey
  • Name – Search Parameter name
  • Type – type for search parameter from spec. Note that the code that uses the indexes needs to know this value implicitly, so I’ve never actually used this field for anything


An actual index value. These are populated whenever a resource is inserted into FhirVersions. All existing entries for the resource are deleted, and a new set are inserted by the code processing the resource (I write this code by hand):

  • EntryKey – internal primary key, never used anywhere yet
  • IndexKey – reference to FhirIndexes.IndexKey – which index this is an entry in
  • ResourceKey – reference to FhirIds.ResourceKey
  • SpaceKey – reference to FhirSpaces.SpaceKey – see below
  • Value – the actual value. Char(128) – search won’t work for values longer than 128 chars


For a QToken, the search is actually on a pair of strings – typically a URL with a leaf value. FhirSpaces just normalises the first string – entries are added to this table for each new URI seen. This table exists purely to conserve space, though it may be useful for optimising searches that involve partial matches on QToken spaces, if we decide this is necessary (not supported for now)


This is the schema I use to provide a fully functional and fully flexible FHIR server. It’s not necessary to have a schema like this – you can implement a server that provides direct access to an existing database schema that supports an actual application. That’d be fine and makes sense for many/most implementations, but the functionality provided by such an implementation is going to be limited to that allowed by the application – in particular, providing version history and fully functional searching (especially chained parameters) is going to be challenging with that approach (though it makes sense – you provide support for what makes sense to do as driven by your schema). But since I’m developing general purpose middleware (and using it as a reference implementation), I need to do everything, and hence this schema that provides full support.


FHIR and the need for identified resources

One of the fundamental tenets of REST is that resources are unambiguously identified. Specifically, for REST,  the identification is an accessible URL that can be used to address the resource with certainty.

The fact that resources are unambiguously identified and that the reference to them can be handled with confidence is a big part of what makes a RESTful implementation simpler and cheaper than other approaches. I’ve found this to be true in my own implementation work based on it – it’s just so much easier to process the information when it’s explicitly identified. This is very different to existing HL7 approaches, such as v2, or CDA, where information is generally carried anonymously – just the information, with no formal record identification.

To illustrate the difference, consider the v2 PID segment. It has several fields of interest:

3 Patient Identifiers
5 Patient Names
7 Date of Birth
8 Patient Sex


The patient identifiers list includes all the patient identifiers,  each potentially labelled with a type. The type list is a mess, and is focused on human relevant types such as “Medical record number”, “Person number”. There is a type for “Patient internal identifier” but I’ve never seen it used in production, and there’s no explicit rules about it’s behaviour

In FHIR, on the other hand, in addition to having these same fields, there’s an explicit and required concept called the logical id of the resource, which is the URL by which it is identified. There are very specific rules about how this URL is maintained, and these are written into the base RESTful spec part of FHIR. Even if you aren’t using REST (you don’t at all have to use REST with FHIR), the rules about how the logical identifier is handled are still the same. This logical identifier can be used confidently when processing the content – and it makes it oh so much easier to handle the content (though it’s not simple – there’s still issues about how records are maintained and synchronised in distributed systems, these haven’t gone away. But their manifestation is now explicit).

One consequence of this is that in order to play with FHIR, you have to pay a price at the entrance – you have to identify your resources. Once you’ve done that, you’re welcome to play, and things are much easier. But if you don’t pay your gate fee… you can’t play.

But what if you can’t afford to pay it?


This is a problem for FHIR that manifests in the XDS implementation we are working on for the connectathon. In a properly defined RESTful implementation of XDS (at least from the FHIR perspective), we should make the resource design as granular as possible, and re-use existing shared resources for basic concepts as much as possible. So we should use Agent and/or person for the author of a document. In this case, the XdsEntry resource has an element author, which refers to another resource that has the details.

The problem is that in classic XDS, the only required item for the author is a name. Just the name. And a name is not enough to identify a person – they can change, and they can be duplicated. The name field is an HL7 v2 XCN, which has an identifier (component 1), but this is often not populated, or if it’s populated, the quality of the identifier is not well described, and it’s unclear to what extent you can rely on it. So in XDS, an author is pretty much unidentified.

This gives you a problem if you are writing a bridge from XDS.b to a FHIR XDS implementation: you have to identify the author (pay the gate fee) as you convert from XDS.b to FHIR – and you usually can’t because you don’t have enough information. Well, perhaps you can, if you know the identifier is good, or you can connect back to the source system to query it for the identification – but that’s going to be custom solution per document provider, and simply isn’t feasible. Alternatively, you can construct phantom identities using UUIds (just give every record a new UUID) – but this just creates reams of duplicate records (millions, in some instances)

Unidentified resources

So this is a real problem – and it’s not about XDS, or unique to XDS. In fact, it crops up anywhere where you want to migrate content from v2, CDA to FHIR as well. And it crops up anywhere you are connecting content to FHIR – the gate fee has to be paid.

But sometimes, it’s just too expensive to pay, such as when you are writing middleware, and you don’t have access to the source records.

What can we do about this in FHIR? Do we need to do anything about it?

One option is simply to stick to the principle, and insist that the gate fee has to be paid, and everything has to be identified. I like this idea when I’m inside the gate – it makes everything so simple. But when I’m outside the gate, it doesn’t look like such a good idea. And I see that other people implementing FHIR have the same experience.

An alternative is to identify the kind of information that you might have if you don’t have a properly identified resource, and allow either a resource reference or some minimal details. You can see that pattern in the contact person for organisation, where it might have a name, address, telecom details, or it might be a reference to an actual person that contains the same details with identity.

But this solution doesn’t scale. Firstly, it creates duplication between the one place and another, and now every implementer has to look in both places. The other problem is that you have to pick and choose – which fields do you allow? So ultimately this is an unsatisfying solution.

An alternative is to allow resources that have no identity. This is a subject that has come up several times on the FHIR mailing list, and again recently amongst the project team and implementers preparing for the connectathon – and it keeps coming up because of the reasons I’ve outlined above.

And in principle, it’s easy. A reference from one resource to another looks like this:


But if you don’t have an identity for the resource, then you could simply replace the content like this:

     <!—- etc -->

The original FHIR draft included exactly this option, but it was removed because allowing this choice complicates implementation in a number of ways:

  • At every resource join, there’s a choice: by reference or by value?
  • XPath processing is much more complicated
  • Server side searching becomes more difficult
  • The schemas become more complex and nested

The biggest issue is that allowing anonymous resources like this allows people to avoid paying the gate fee, and making implementation within the fences more expensive. That’s because there’s a basic rule of thumb: once a resource has been de-identified like this, it’s not feasible to re-identify it, and if an information provider choose not the to the pay price, then the information consumers will have to – and there’s usually more consumers than providers.

I think that we have to allow this choice for resource references – by value or by reference. I’ll be making a ballot comment on FHIR that  we should, and implementing this choice for the connectathon so that we can see how it works in practice.

Question: Special Characters in HL7 v2


Can Hl7 v2 messages contain any special characters in them?


A standard v2 message is restricted to ASCII characters in the range 32 to 126. Some of the characters need to be use for syntactical separators, and so there is special support for escape sequences to represent these actual characters in the message. But these sequences don’t support special characters.

To support special characters, you can do one of two things:

  • Use a unicode message (i.e. UTF-16). Note that using unicode can only be done by trading partner agreement, and not many systems support unicode (mostly they are still ASCII back ends, though unicode support is wider in Europe and Asia than in primarily english speaking countries, where systems are migrating to unicode very slowly)
  • Use a character escape sequence – \M or \C. These are even less supported, and not well understood (see chapter 2 of the v2 standard)

Alternatively, you can just pretend that a code page agreement has been made, and use ascii characters in the range 127 – 254. This is infrastructurally easy, but hard to manage – code pages are poorly understood, often unmanaged, and translation between them is often impossible.  (Naturally, then, this is what I see most often in practice)


Remote Participation in the FHIR Connectathon

We’ve had a request to participate in the FHIR connectathon remotely. In other words, people would like to participate in the Connectathon, but won’t be in Phoenix.

Is that possible? Well, maybe. At the last connectathon, most – but not all – servers were in the cloud. And given that the cloud servers are generally available continuously anyway, the real point of the connectathon is that

  • People are fully focused on the task at hand
  • The dynamics of a face to face group meeting

Remote participants miss out on the second – and it’s important. So some degree, video conferencing via skype or something may fill some of the gap – but generally these techniques are very poor substitutes for actually attending.

So we’re thinking about whether we’ll support remote participation, and if so how. But one factor in the deliberations is how many people want to participate like that – we don’t know.

If you think that you would like to participate in the connectathon, but can’t actually be in Phoenix – can you drop me a line – thanks

Implementing ATNA on a FHIR server

This is the first of a series of posts that I’ll be making about the trial implementation of the IHE XDS profile using FHIR resources. Note that the status of the FHIR resources for XDS is a strawman – both HL7 and IHE are looking at these and evaluating their suitability for use for providing a mobile friendly gateway to an XDS system, but neither organization has committed to this approach yet – we have to see how well it works.

The first step in the implementation is to provide a working security logging framework since this is a fundamental requirement for all XDS actors. This post describes the way that ATNA (rfc 3881) is implemented using FHIR resources.

Firstly, a quick primer for rfc 3881: a security log message is sent from an application to a specialized security logging server that provides logging and analysis services. The security log message is sent using either UDP or secure-TCP, with a specified xml format at the end – the XML part is what is specified in RFC 3881. The XML specifies the kind of event the took place, information about the user and system context in which it takes place, and information about what system objects were affected by the operation – and whether the operation succeeded or not. The ATNA actors describe a simple flow – event records are created on one system, and sent to the other system for storage. No other interactions are allowed.

In order to reproduce this functionality, the FHIR specification defines the “SecurityEvent” resource. This resource includes a few of the fields from the security event log itself and the content specified by RFC 3881. In a RESTful context, the source system creates the resource – using either XML or JSON, and posts it to the security logging system using an HTTP post (a FHIR “create” event).The HTTP post can be secured using SSL to reproduce the same functionality as the secure-TCP syslog equivalent.

A RESTful approach based on FHIR is better because:

  • syslog is alien to many developers these days, while HTTP/SSL libraries are ubiquitous
  • There are many HTTP/network configuration, debugging and manipulation frameworks and services
  • JSON is defined out of the box, and equivalent and interconvertable with the XML format
  • The exchange is easy to extend and leverage

The last point requires some clarification. To start with, the “create” function is only one of many defined by FHIR – there is also update, and delete, though these would generally by disabled in a security logging system, or at least highly restricted. There is also the ability to read existing security logs, search them, and use pub/sub to replicate some or all of the events on other systems. These operations are not the basis for a full security event analysis service, but do open up the system for more reuse.

In addition, FHIR defines many other functions that can be used, such as profiles and conformance statements, and also offers the ability to bundle the security information about the operation along with the operation itself – that resolves a conundrum with the existing ATNA spec – do you log before you find out whether the operation works, or afterwards, when you might not be able to log it? Other FHIR framework advantages also apply, such as RDF based mappings into a ontological reasoning systems. It would be reasonable to expect that at some point in the near future this would be a useful addition to security analysis services.

Trying it out

There’s a public FHIR server offering ATNA functionality. You can get a list of security events on the system as html, xml or as JSON, or you can create new events on the system by posting the appropriate XML or JSON to (or https:, though the server uses a self signed certificate). The system doesn’t allow updates or deletes to security events (as defined in the conformance profile here).

To assist with testing, this server offers a IHE-certified ATNA server on port 3099 of (also known as Both UDP and secure TCP are offered (secure TCP uses the trusted certificate issued to Health Intersections from Gazelle). Any ATNA Audit Records sent to this server will be added to the FHIR repository as SecurityEvent resources, and will show up in the feeds.

Anyone is welcome to use this test server to work up a FHIR Security Event logging or handling system. I’d especially appreciate feedback with regard to the performance of the ATNA-FHIR bridge – I’m not sure how solid the conversion of codes is.

FHIR Connectathon

The next FHIR connectathon is focused on XDS. Because we are focusing on the interfaces, not the system functionality, production of security events is not required. But I encourage everyone to implement these anyway – this is pretty simple stuff once you have done anything with FHIR.


FHIR Webinar – reference and additional answers

This week I did a webinar on FHIR for HL7. They’ve posted the recording of it (2012 December 4 Ambassador Webinar: Fast Healthcare Interoperability Resources (FHIR) (70.26 MB)), along with my slides (PDF version of the slides here).

There were a number of questions asked that we didn’t have time to get to. Here’s some quick answers:

“Q: Is this planned to replace all v2.x and v3 standards, and if so, when?

A: Well, it might. If it it turns out to be as easy as we hope, then this will happen, slowly, and of it’s own accord. But HL7 will – as it should – be very careful about phasing out well respected widely implemented standards in favor of something new – the evidence and market selection would need be overwhelming.

“Q: When do you envision FHIR starts being commercially used in the marketplace?

A: This coming year, actually. Though as I said in the webinar, not in areas where v2, CDA, etc are already incumbent. I think XDS may well be first.

“Q: Can you show an example, end-to-end, from initiating the query, to the response?

A: Ummm, not off hand, no as a canned example. But I can do something better than that – here’s a search that returns all the persons named “peter” (as html, as xml, as json). You can fool with the parameters, or run it again from here. And use something like fiddler or proxytrace to see the actual traffic between your browser and the server, which is the same as the traffic from an application.

Q: when/where is the next connectathon?

A: Connectathon details here.

Q: At connectathon, what is scope of objects that will be considered? i.e., if we are modelling prospective orders for a particular IV set with certain additives, could that be an appropriate topic?

A: For reasons of practicality, we are focusing on XDS this time. We’d love to focus on an order scenario next time around.

“Q: How does the introduction of FHIR affects CDA implementation Guides? Will the guides be obsolete if FHIR is adopted as a standard?

A: See answer above about existing standards.

“Q: How does FHIR work together with IHE/XDS profiles?

A: That’s really worth a whole post. But the notion is that XDS.b is a transactional view of a document repository, where as FHIR will offer a RESTful view of the repository. I can imagine a true XDS.b server hosting a FHIR interface, and also a true FHIR server hosting an XDS.b interface (I’m going to be implementing a FHIR server that offers RESTful access to documents in folders, and accepts submissions by XDS.b ProvideAndRegisterDocument. The FHIR interface is closely aligned to the candidate MHD interface, but resolves some unanswered questions in the MHD profile, and is less transactional again.

“Q: is FHIR trying to be developed in android environment or is it still to early to define the software platform for implementation?

A: Not at all too early. We already have Android applications based on java code generated from the various artifacts in the specification, and we’ll be hoping for more at the next connectathon.

“Q: Specific license language/example would be great ie: GNU, Eclipse License, etc

A: see The generated code is licensed under BSD (as the most permissive license of all that we could find). Standards are not software applications, and the licenses that are suitable for software don’t work for standards.

“Q: In an outpatient diagnostic center, we have many referring doctors that in order to send and receive orders and reports, need to implement a costly HL7 interface for up to $10,000 each. Can FHIR make this process simpler and cheaper then building an HL7 interface for up to $10,000 each?

A: I’d really like it to be much cheaper. But when it comes to ordering, much of the complexity is inherent in the problem, not the syntax or exchange paradigm, so FHIR isn’t a magic bullet.

“Q: I’m interested in understanding how you see Interface Engines making use of the FHIR standard. Is the ideal to transform it to FHIR xml over MLLP or to receive HL7V2 and then post atom to subscribers through HTTP?

A: Actually, I write an interface engine, and all my implementation is in that context. I’ll be offering bridges between v2 and CDA and XDS and FHIR/REST and also FHIR messaging with transforms between them. Of  course, the transforms will generally be custom ones, particularly for v2…

“Q: I utilize HL7 for LAB, HIPPA claims revenue cycle between insurance Payers and Healthcare facilities as well as feeding vendors and regulatory based on Trading Partner Agreements and Confidentiality guidelines. Is there a conversion chart for HL7 eExchange from the current standard use to FHIR?

A: Not yet, because we haven’t covered a lot of that ground in FHIR yet. Hopefully this will come.

Q: Data Repository feeds as well

A: Well, generally, FHIR is particularly suitable for feeds – just use pub/sub using atom

Q: Thank you for presenting…This has been very informative and I see great potential

A: –unanswered–

“Q: Seems like you are implementing ‘linked data’ concepts

A: Err, yes, we’re keeping an eye on that work as well

Q: Congratz, this seems more intuitive than HL7v3 or CDA and hopefully it will be adopted like “”FHIR”” 🙂

A:Thanks – that’s what’s driving this. If we get it right, it will be more intuitive than v2 as well

Question: How does the HL7 v3 Dynamic Model work


What is the actual usage of HL7 dynamic model? The interaction defines trigger event, composite message type, and receiver responsibilities. But does it really matter in message exchange? How are trigger events and receiver responsibilities expected to be used?


Trigger events provide context about when interactions are expected to be exchanged.  For example, a particular notification interaction might be expected to be sent when the state of an object changes to “completed”.  In theory, a system that fails to initiate an interaction when the event described by the trigger event occurs could be considered non-conformant.  However, the trigger event alone isn’t sufficient for an interaction to flow.  The application also needs to have an appropriate recipient defined and meet other business rules such as access permissions.  As well, there are no rules about exactly how quickly after the occurrence of the event the interaction needs to actually be sent over the wire.  The best that can really be done is to confirm that a compliant system is *capable* of sending the interaction in response to the defined triggering condition.

Another consideration is that some interactions may share the same trigger event.  Frequently this is a acknowledgement interaction and a notification interaction, both triggered by a state transition caused by some sort of request interaction.  This means that if a system supports both interactions, it would be expected to invoke both kinds when the event occurs.

Finally, trigger events give a little bit of insight into expected application behaviour.  If the trigger event is a state transition, then that identifies that the application actually changes the state of the specified object type.

Receiver responsibilities are more important.  They define the “application acknowledgement” that is expected in response to a given interaction.  For example, an interaction that requests that a prescription be suspended might have receiver responsibilities to either change the state of the prescription (fire a trigger event) and send an acknowledgement interaction indicating the prescription has been changed or, alternatively, to send a “refusal” interaction indicating that the state of the prescription wasn’t changed and possibly indicating why.

Most commonly receiver responsibilities follow simple patterns.  Query interactions have a responsibility to send a response.  State transition requests and fulfilment requests typically have alternate receiver responsibilities for both acceptance and refusal.  Notifications generally have no receiver responsibilities.  They might receive back an accept acknowledgement, but no application-level acknowledgement is expected.

However, receiver responsibilities can get more sophisticated.  For example, a request to fulfil an order might be met with a counter-proposal interaction that says “I can’t do that, but I could do this instead”, which might in turn have a receiver responsibility to agree or refuse.  These patterns are rare, in part because they don’t work well in synchronous architectures.

HL7 has been looking for alternative dynamic model that provides broader capabilities, such as the ability to indicate that a lab system, after promising to fulfil an order also has a responsibility to later send the results, or perhaps a cancellation of their promise indicating they were unable to fulfil the test (lost the sample, etc.)  However, it’s been challenging to do this in a way that doesn’t make v3 more overwhelmingly complex than it already is, both for committees designing the specifications and the implementers who must realize them.

Note: Lloyd McKenzie provided this answer – thanks Lloyd