Category Archives: Programming

#FHIR and R (Stats / graphing language)

I’ve spent the last 2 days at the 2017 Australian R Unconference working on an R client for FHIR.

For my FHIR readers, R is a language and environment for statistical computing and graphics. (Having spent the last couple of days explaining what FHIR is to R people).  My goal for the 2 days was to implement a FHIR client in R so that anyone wishing to perform statistical analysis of information available in R could quickly and easily do so. I was invited to the R Unconference by Prof Rob Hyndman (a family friend) as it would be the best environment to work on that.

My work was a made a lot easier when Sander Laverman from Furore released an R package to do just what I intended to do earlier this week. We started by learning the package, and testing it. Here’s a graph generated by R using data from test.fhir.org:

Once we had the R Package (and the server) all working, I added a few additional methods to it (the other read methods). For sample code and additional graphs, see my rOpenSci write up.

I think it’s important to make the FHIR data easily available to R because it opens up a connection between two different communities – that’s good for both. Many of the participants are the Unconference were involved in health, and aware of how hard it is to make data available for analysis

Restructuring the data

FHIR data is nested, heirarchical, and focused on operational use. I’ve written about the difference between operational and analytical use before. Once we had the data being imported from a FHIR server into a set of R data frames, Rob and I looked at the data and agreed that that most important area to focus on was tools to help reshape the data down to a usable form. The thing about this is that it’s not a single problem – the ‘usable form‘ will depend entirely on what the question that is being asked of the data is.

So I spent most of the time at the Unconference extending my graphQL implementation to allow reshaping of the data (in addition to it’s existing function in assembling and filtering the data). I defined 4 new directives:

I’ve documented the details of this over on the rOpenSci write up, along with examples of how they work. They don’t solve all data restructuring problems by a very long shot, but they do represent a very efficient and reusable way to shift the restructuring work to the server.

There was some interest at the unconference in taking my graphQL code and building it into an R package to allow graphQL query of graphs of data frames in R, to assemble, filter, and restructure them – it’s an idea that’s useful anytime you want to do analysis of a graph of data. But we were all too busy for that – perhaps another time.

Where to now?

I think we should add R support to the AMIA FHIR datathon series, and maybe it’s time to encourage the research track at the main FHIR connectathons to try using R – I think it’s a very powerful tool to add to our FHIR toolkits.

Thanks to Adam Gruer from Melbourne’s Royal Children’s Hospital for helping – those graphs are his. Thanks also the organisers – particularly Nick Tierney (gitmeister estraordinaire). I picked up some ideas from the Unconference that will improve the FHIR connectathons.

Verhoeff Algorithm in Delphi

I needed an implementation of the Verhoeff check digit algorithm in Delphi, and couldn’t find one. So reproduced here for ease of use by anyone else who needs it:

const
 verhoeff_d : array[0..9,0..9] of integer = (
 (0, 1, 2, 3, 4, 5, 6, 7, 8, 9),
 (1, 2, 3, 4, 0, 6, 7, 8, 9, 5),
 (2, 3, 4, 0, 1, 7, 8, 9, 5, 6),
 (3, 4, 0, 1, 2, 8, 9, 5, 6, 7),
 (4, 0, 1, 2, 3, 9, 5, 6, 7, 8),
 (5, 9, 8, 7, 6, 0, 4, 3, 2, 1),
 (6, 5, 9, 8, 7, 1, 0, 4, 3, 2),
 (7, 6, 5, 9, 8, 2, 1, 0, 4, 3),
 (8, 7, 6, 5, 9, 3, 2, 1, 0, 4),
 (9, 8, 7, 6, 5, 4, 3, 2, 1, 0)
 );
// The permutation table
 verhoeff_p : array[0..7,0..9] of integer = (
 (0, 1, 2, 3, 4, 5, 6, 7, 8, 9),
 (1, 5, 7, 6, 2, 8, 3, 0, 9, 4),
 (5, 8, 0, 3, 7, 9, 6, 1, 4, 2),
 (8, 9, 1, 6, 0, 4, 3, 5, 2, 7),
 (9, 4, 5, 3, 1, 2, 6, 8, 7, 0),
 (4, 2, 8, 6, 5, 7, 3, 9, 0, 1),
 (2, 7, 9, 3, 8, 0, 6, 4, 1, 5),
 (7, 0, 4, 6, 9, 1, 3, 2, 5, 8)
 );
 //The inverse table
 verhoeff_inv : array [0..9] of char = ('0', '4', '3', '2', '1', '5', '6', '7', '8', '9');

//For a given number generates a Verhoeff digit
function genCheckDigit(s : String): char;
var
 i, c, len : integer;
begin
 c := 0;
 s := s + '0';
 len := length(s);

 for i := len downto 1 do
   c := verhoeff_d[c][verhoeff_p[((len-i) mod 8)][ord(s[i]) - ord('0')]];

 result := verhoeff_inv[c];
end;

////Validates that an entered number is Verhoeff compliant.
////The check digit must be the last one.
procedure validateCheckDigit(s : String);
var
 i, c, len : integer;
begin
 c := 0;
 len := length(s);

 for i := len downto 1 do
   c := verhoeff_d[c][verhoeff_p[((len-i) mod 8)][ord(s[i]) - ord('0')]];

 if c <> 0 then
 raise Exception.Create('Check digit error: "'+s+'" is not valid by Verhoeff algorithm');
end;

 

Announcement: #FHIR Code-a-thon Washington DC April 1/2

An announcement from CMS/HHS Entrepreneur-in-Residence and FHIR community member Mark Scrimshire:
I wanted to let you know that we are preparing a FHIR Code-a-thon in Washington DC on Friday April 1st and Saturday April 2nd. We have Susannah Fox opening the event and the Code-a-thon will see the first public access to a prototype CMS Blue Button on FHIR data API. We are busy readying a synthetic data set to make available through the API.
It would be great if you could let fellow Argonauts, and other interested parties, know about our event. Advanced registration is required. People can sign up at http://HealthCa.mp/onfhir/signup
The winners of the FHIR Code-a-thon will receive tickets to the Health Data Palooza where they will have the opportunity to demonstrate their winning solution.

 

I’d say that I’d see you there, but I’m afraid I’ll be surfing in Samoa on those days  – sorry.

Question: Locating the #FHIR end-point

Question:

I have been trying to develop FHIR app using your API. The url “http://fhir2.healthintersections.com.au/Patient” doesnt return me list of patients. Am i hitting the right URL?

Answer:

No. and in fact, this trips lots of people over for lots of servers. Very often, the link provided to the server is a reference to a web page that describes the various end-points the server offers, and the security details for them. For instance, my server offers 2 end points – http://fhir2.healthintersections.com.au/open, which anyone can use, and https://fhir2.healthintersections.com.au/closed, which requires authorization via Smart on FHIR.

So the url you’re looking for is http://fhir2.healthintersections.com.au/open/Patient

 

Right-To-Left scripts in #FHIR

Today, I’m at Linux Conference Australia. I watched a hysterically funny and very interesting presentation by Moriel Schottlender on using Right-To-Left scripts, such as Hebrew, Arabic, Farsi, etc:

browsers have problems when deciding what to do when languages are mixed up, and that, my friends, is a recipe for really weird issues when typing and viewing bidirectional text.

Check the full presentation. And yes, Moriel showed us some really confusing stuff. She kept showing images like this:

This, btw, is your mind trying to process RTL issues (and how could I not post that image?). Moriel has a web site demonstrating some of this stuff (including the piece de resistance, an animated bmp).

Anyway, after the presentation, I asked Moriel whether she had any thoughts about RTL use in a RESTful interface (say, maybe, FHIR). She didn’t have any strong experience, but a combination of her presentation, and her comments after, led to the following advice:

  • If you’re using RTL script entirely, that should be pretty straight forward. The framework of a resource is all LTR ASCII, but the contents of the string primitives are RTL. That works fine
  • The tricky bit is where you mix scripts – e.g. an Arabic word mixed in with an english sentence

Actually, that’s not at all an unusual idea. You can see an example of this here on Wikipedia. And in a clinical environment, it would be a typical thing to happen in somewhere like Saudi Arabia, with a lot of foreigners working in the healthcare system, and mixing English and Arabic. You can watch Moriel’s presentations for fun with left, right, parentheses etc, and see how tricky that could be. Anyway, the basic advice is:

Anywhere you mix LTR and RTL scripts, make sure you use Unicode control characters to mark the edges of the direction changes.

And one more conclusion I drew:

  • There’s little prospect we’ll ever publish an IG that makes sensible use of RTL content

If anyone has experimented with RTL content in FHIR – I’d love to hear about it: we want to support that properly.

p.s. I’m here at LinuxConf to talk about the FHIR community as an open source project.

#FHIR and HTTP Patch

At the last connectathon, we had a stream testing support for PATCH using JSON patch. A number of us participated in the stream, and some of us got JSON patch working fine.

But it’s not clear that JSON Patch is the solution we’re looking for.  JSON Patch has some advantages:

  • It’s an external standard – we like that
  • It’s pretty simple – we like that too
  • There’s existing libraries, and set of test cases – we really like that

But. The problem is that JSON patch is based on JSONPointer – when you specify changes in a list of objects, you specify the list by the numerical offset of the object in it. This means that in order to make a set of changes to a set of JSON objects, you must know exactly what the existing content is. So in order to use JSON patch, you must first query a resource for it’s latest state, determine the changes that you wish to make, and apply them using a version specific  update.

Sure, that sequence works. But it doesn’t really deliver on the 2 key use cases that people have identified for using PATCH in FHIR:

  • Save a client from fetching an entire resource to make updates to it
  • Manage more effectively updates when the client doesn’t have access to all the parts of a resource

The second case is interesting. The problem is that when a server gets an update from the client, to figure out how to use it, it must determine what the client has access to; if the client hasn’t get access to it, it can’t delete content that the client hasn’t mentioned in the update. Tricky. But because JSON Patch using list index offsets, it turns out that the basic processing logic isn’t any different – the server must determine what the client has access to in order to figure that out.

There’s one other problem with the JSON Patch approach – it doesn’t support XML, and there’s lots of FHIR implementers using XML.

FHIR Patch

There’s an alternative approach that might be better. Instead of using index based offsets, allow a general query. And use the FHIR parameters infrastructure, to support both JSON and XML.

I would propose a smaller set of operations, but they have different parameters:

  • Add – content to add
  • Remove – selection query (using FHIRPath)
  • Replace – selection query (using FHIRPath) + content to add
  • Test – query that must evaluate to true

Note: I’ll be making another blog post shortly about where FHIRPath is up to. 

This is more capable than the JSON patch approach – you can specify a change to make without first having to get the resource. For instance, you could say, if the patient doesn’t have this phone number, then add it.

But this has some disadvantages too:

  • there’s no external libraries support for this
  • it needs FHIRPath. while I think that everyone will eventually need to support this, so far, there’s no particular need to for a lot of implementers
  • It’s much more complex to test

There was some discussion around this during the connectathon, but there was no clear preference amongst the participants. None of use like these disdvantages, but we don’t really like the limitations either. Comments are welcome.

 

Improved Documentation of the $closure operation for #FHIR Connectathon 11

One of the stated goals of FHIR Connectathon 11 in Orlando in a few weeks time is for the terminology services stream participants to test out the $closure operation.

The $closure operation is an important one because it enables terminology consumers to close the loop – to integrate terminological based reasoning into their internal search capabilities, without having to split the search across multiple technologies – which is a real challenge. For additional documentation about the closure table process, see http://hl7.org/fhir/terminology-service.html#closure

However, up to now, the terminology connectathons have focused on other parts of the terminology services, and $closure hasn’t been tested – or even implemented.

In preparation for the Connectathon, I’ve implemented the $closure operation. Here’s some additional API documentation for the $closure operation, which will be the basis for the connectathon. Note that all this documentation is in JSON, but XML should work the same for servers that support XML.

Note: this is additional API documentation for connectathon implementers. Before using the specific documentation below, you should be familiar with the $closure operation concepts and description.

To help connectathon participants, I’ve published both a client and server that implement this operation. The server is at http://fhir2.healthintersections.com.au/open, and the client is in my ValueSet editor (Tools…Closure Manager). Note that the value set editor is still rough – I haven’t got the rest of it up to speed for DSTU2 quite, but the closure table part works.

Initialising a Closure Table

Before it can be used, a closure table has to be initialised. To initialise a closure table, POST the following to http://fhir2.healthintersections.com.au/open/ConceptMap/$closure:

{ 
  "resourceType" : "Parameters", 
   "parameter" : [{ 
     "name" : "name", 
     "valueString" : "eaab9a21-db9a-45e0-a400-0075eb19f795" 
  }]
 }

A successful response from the server looks like this:

{
 "resourceType": "Parameters",
 "parameter": [
    {
      "name": "outcome",
      "valueBoolean": true
    }
  ]
}

If there is an error – usually involving the closure name, the server returns a HTTP status 400 with an operation outcome:

{
  "resourceType": "OperationOutcome",
  "text": {
    "status": "generated",
    "div": "<div xmlns=\"http://www.w3.org/1999/xhtml\"><p>invalid closure name \"invalid-id!\":</p></div>"
  },
  "issue": [
    {
      "severity": "error",
      "details": {
        "text" : "invalid closure name \"invalid-id!\""
      }
    }
  ]
}

Note: As a matter of policy, my server requires valid allows UUIDs for closure names unless you log in using SMART on FHIR first, in which case you can use any valid id value for the closure name, but the closure is unique to the login behind the OAuth unless it’s a UUID. Other servers may choose other policies.

Adding to a closure Table

When the consumer (client) encounters a new code, it POST the following to http://fhir2.healthintersections.com.au/open/ConceptMap/$closure:

{
  "resourceType" : "Parameters",
  "parameter" : [{
    "name" : "name",
    "valueString" : "eaab9a21-db9a-45e0-a400-0075eb19f795"
  }, {
    "name" : "concept",
    "valueCoding" : {
       "system" : "http://snomed.info/sct",
       "code" : "22298006",
       "display" : "Myocardial infarction"
    }
  }]
}

Note that this example only includes one concept, but more than one is allowed:

{
  "resourceType" : "Parameters",
  "parameter" : [{
    "name" : "name",
    "valueString" : "eaab9a21-db9a-45e0-a400-0075eb19f795"
  }, {
    "name" : "concept",
    "valueCoding" : {
       "system" : "http://snomed.info/sct",
       "code" : "22298006",
       "display" : "Myocardial infarction"
    }
  }, {
    "name" : "concept",
    "valueCoding" : {
       "system" : "http://snomed.info/sct",
       "code" : "128599005",
       "display" : "Structural disorder of heart"
    }
  }]
}

The response varies depending on the conditions on the server. Possible responses:

If the closure table has not been initialised:

Return a 404 Not Found with

{
  "resourceType": "OperationOutcome",
  "text": {
    "status": "generated",
    "div": "<div xmlns=\"http://www.w3.org/1999/xhtml\"><p>invalid closure name \"eaab9a21-db9a-45e0-a400-0075eb19f795\":</p></div>"
  },
  "issue": [
    {
      "severity": "error",
      "details": {
        "text" : "invalid closure name \"eaab9a21-db9a-45e0-a400-0075eb19f795\""
      }
    }
  ]
}

If the closure table needs to be reinitialised:

Return a 422 Unprocessable Entity with

{
  "resourceType": "OperationOutcome",
  "text": {
    "status": "generated",
    "div": "<div xmlns=\"http://www.w3.org/1999/xhtml\"><p>closure \" eaab9a21-db9a-45e0-a400-0075eb19f795\" must be reinitialised</p></div>"
   },   
   "issue": [{
       "severity": "error",
       "details": {
         "text" : "closure \" eaab9a21-db9a-45e0-a400-0075eb19f795\" must be reinitialised"
       }
     }
   ]
}

The server should only send this when it’s underlying terminology conditions have been changed (e.g. a new version of SNOMED CT has been loaded). When a client gets this, it’s only choice is to initialise the closure table, and process all the codes in the closure table again (the assumption here is that the system has some external source of ‘all the codes’ so it can rebuild the table again).

If the concept(s) submitted are processed ok, but there’s no new concepts, or no new entries in the table, return a 200 OK with :

{
    "resourceType": "ConceptMap",
    "id": "42dc941b-7ff9-4859-a4cb-9592007a26e5",
    "version": "1",
    "name": "Updates for Closure Table eaab9a21-db9a-45e0-a400-0075eb19f795",
    "status": "active",
    "experimental": true,
    "date": "2015-12-20T23:12:55Z"
}

Note that in the $closure operation, the response never explicitly states that a code is subsumed by itself. Clients should assume that this is implicit.

If there’s new entries in the closure table:

The server returns a 200 ok with:

{
    "resourceType": "ConceptMap",
    "id": "b87db127-9996-4d0c-bda9-a278d7a24a69",
    "version": "2",
    "name": "Updates for Closure Table eaab9a21-db9a-45e0-a400-0075eb19f795",
    "status": "active",
    "experimental": true,
    "date": "2015-12-20T23:16:24Z",
    "element": [{
        "codeSystem": "http://snomed.info/sct",
        "code": "22298006",
        "target": [{
           "codeSystem": "http://snomed.info/sct",
           "code": "128599005",
           "equivalence": "subsumes"
        }]
    }]
}

Note: it’s important to get this the right way around. From the spec: The equivalence is read from target to source (e.g. the target is ‘wider’ than the source). So in this case, 128599005 (Structural disorder of heart) subsumes 22298006 (Myocardial infarction).

Notes:

  • The server can return multiple elements, each with 1 or more targets
  • servers may return the relationship represented in either direction.

Re-running a closure operation

The way that the closure operation functions, it’s possible for a client to lose a response from the server before it is committed to safe storage (or the client may not have particularly safe storage). For this reason, when a client is starting up, it should check that there has been no missing operations. It can do this by passing the last version it is sure it processed in the request:

{ 
  "resourceType" : "Parameters", 
   "parameter" : [{ 
     "name" : "name", 
     "valueString" : "eaab9a21-db9a-45e0-a400-0075eb19f795" 
  }, {
     "name" : "version", 
     "valueString" : "3" 
  }]
 }

That’s a request to return all the additions to the closure table since version 3. The server returns its latest version in the concept map, along with anything added to the closure table since version 3 (not including version 3)

Notes:

  • The client can pass a concept or version, but not both
  • These examples (and my server) use a serially incrementing sequential integer, but this is not required, and clients should not assume that there is any meaning or order in the version. Just recall the last version, and treat it as a magic value. There is, however, one special value: ‘0’. Passing a last version of 0 should be understood as resyncing the entire closure table

 

A Path expression language for #FHIR

Since the beginning of working with FHIR, we’ve had a number of contexts in the specification where we’ve need to refer to a particular element within a resource, or make rules on the properties of the elements in a resource. Over time, that list has grown, and within the specification we now have the following places for this:

  • search parameter paths – used to define what contents the parameter refers to
  • invariants in ElementDefinition, used to apply co-occurance and other rules to the contents
  • slicing discriminator – used to indicate what element(s) define uniqueness
  • error message locations in OperationOutcome

In addition, Smart on FHIR’s cds-hooks has a template approach which really needs something like this, and we have an identified need for patch which needs something like this. In addition, implementations may find other uses for this as well – I know I have several places where I need to refer to content in a resource dynamically. One really important context is a formal mapping specifications

Up to now, we’ve used a mix of XPath and informal dotted notations. Feedback from implementers is that this is a barrier to adoption, and we need a better approach. XPath is a problem because:

  • XPath is XML specific
  • We’ve used XPath2 features in our XPaths, and XPath2 is very poorly supported to the point
  • XPath is overkill for the simply use cases, and too hard for the complex use cases

As for JSON, there’s a raft of variations around “XPath for JSON”, but none of them seem to be on any standards path. And what we really need is a single language that works for both XML and JSON, and is easy to use for simple cases, and powerful enough for the difficult cases. It become clear, working with implementers after the publication of DSTU2, that it was time to address this. And it was clear that we need to define our own language for this (this is contentious – surely, people say, there is already something out there for this, so we waited for 3 years before acting on this, but eventually we couldn’t wait).

So we’ve defined a little expression language for these purposes. Several of us have implemented processing engines for it, to validate the approach. It’s not part of the formal specification right now, but it’s certainly something we expect to make part of it.

Useful Links:

Note: we’ve not formalised any of this yet – including the name “FHIRPath”. Discussion and suggestions on names, syntaxes, and approaches is all welcome

 

FHIR Notepad++ Plug-in: Tools for #FHIR developers

I’m pleased to announce that the FHIR Plug-in for Notepad++ that was distributed and tested at the DevDays in Amsterdam last week is now ready for general release.

Notepad++ is a powerful text editor that’s pretty popular with developers – it seems most of us who use windows use it. And it has a flexible and powerful plug-in framework with an active community around that. So it was the logical choice for a set of FHIR tools. The FHIR tools themselves offer useful functionality for FHIR developers (programmers, analysts), based on the kinds of things that we need to do at connectathons or for authoring content for the specification.

ToolBox

toolbar

The FHIR Toolbox makes the following functionality available:

  • XML / JSON interconversion
  • Interactive and background resource validation
  • FHIR Path execution and debugging (more on that in a subsequent post)
  • Create a new populated resource from its definition (or a profile on it)
  • Connect to a server (including Smart-on-FHIR servers)
  • Create a new populated resource from its definition (or a profile on it)
  • Open a resource from a server
  • PUT/POST a resource to a server
  • Narrative generation

Visualizer

visualizer

In addition, there’s a visualizer provided. This shows the following kind of information:

  • Narrative for the resource
  • Validation and Path execution results
  • Information about the current element (e.g. viewer for attachments, codes, references)

Note that the FHIR Plug-in doesn’t provide syntax highlighting for XML or JSON – there’s other plug-ins for them.

Additional Notes:

  • The FHIR tools are based on DSTU2
  • There is ongoing active development of the plug-in, so expect glitches and bugs (report bugs using my FHIRServer Github project)
  • To get notified of updates, subscribe to the RSS link from the downloads page. Or the plug-in itself will notify when a new version is released
  • yes, Notepad++ is only available for windows. For OSX developers, well, you can get a windows VM. It’s unlikely that it will be feasible to port the tools to OSX
  • For further documentation, see the Wiki page

 

#FHIR DevDays Amsterdam 2015

In two weeks time – Nov 18 2015 – I will be joining Ewout Kramer from Furore in Amsterdam – along with other core team members James Agnew, Josh Mandel, Lloyd Mckenzie – for the Furore #FHIR DevDays 2015. I’m really looking forward to this – it’s the peak European FHIR event, and we’ve got a great program lined up:

  • Patient Track: Create, update and search patients with FHIR.
  • Terminology Services Track: See if you can work with FHIR’s terminology operations: expand valuesets, validate your codes and get human readable labels for your codes.
  • Profile & Validation Track: Create a profile and an instance, and ask a server to validate the instance according to your profile.
  • SMART on FHIR track: Extend your server or build a client to add OAuth2 to the FHIR REST interface, and see whether they can authenticate and talk.
  • Imaging Track: Imaging, DICOM and FHIR.
  • API Beginners Track: Get your very first FHIR client application up and running.
  • Community Track: Presentations of real life experiences with FHIR.

The full schedule is here: http://goo.gl/gHGmCB, along with the the presenters (Keith Boone, Brad Genereaux, Michel Rutten, Dunmail Hodkinson, Mirjam Baltus, Kevin Mayfield, Marten Smits, René Spronk, Simone Heckmann – what a stellar line up!)

In addition to this, the FHIR Core Team will be announcing a couple of new and exciting implementation features for the community at DevDays – hopefully I’ll see you there!