FHIR Chat · Messaging to REST API conversion · v2 to FHIR

Stream: v2 to FHIR

Topic: Messaging to REST API conversion

Cooper Thompson (Nov 03 2021 at 14:18):

Is the v2 to FHIR group only focused on mapping the v2 data models to FHIR? Or is there also work to map the messaging exchange model to a REST API model? It seems like data model conversion is the easy part. We've seen folks work on "v2 to FHIR" thinking that they just need to map data, but the sticking point is that the intermediary always has to do the query/match/create/update/delete process to convert from snapshot content in a message to a set of interactions on the REST API.

For example, in order to convert GT1 from a message to Account in the REST API, you need to:

1. Gather the list of GT1s from the message.
2. Gather the list of Accounts from the REST API server.
3. Match the GT1 records to Account records.
4. Delete any Accounts on the FHIR server that are not in the GT1 list.
5. Update any Accounts on the FHIR server that are in the GT1 list.
6. Create any Accounts that are in the GT1 list but not on the FHIR server.
7. Roll back if anything in this whole process failed.

Since you need to do this same thing for basically every set of segments (or at least snapshot-mode style segment), this gets really chatty on the wire.

Or were we hoping that v2 to FHIR would be using FHIR messaging?

René Spronk (Nov 03 2021 at 15:08):

Even if we were to use FHIR messaging, the receiving FHIR server (one that supports FHIR messaging) would still have to do all of the above. This is highly contextual business layer stuff, so other than best practices it's unlikely that HL7 will ever provide detailed guidance in this area.

Cooper Thompson (Nov 03 2021 at 17:04):

Yeah - that makes sense to me too. I just think folks (or at least some) have a perception that "v2 to FHIR" is something that will make it relatively straightforward to do a "v2 message" to "FHIR API" conversion.

René Spronk (Nov 04 2021 at 07:37):

The work by HL7 to create a segment-to-resource mapping is tremendously helpful, but even there one will need tweaks because of variability in usage of v2. But the hard parts lie elsewhere.

FHIR to v2 has an entirely different set of challenges assiciated with it, but IMHO in general that'll probably be easier to implement than v2 to FHIR.

Hans Buitendijk (Nov 04 2021 at 14:33):

The purpose of the project is to focus on the data mapping using a FHIR message as the target to ensure all data is considered. Then it is up to the mapper/receiver to determine whether to actually just use to to persist the FHIR resources, continue as a FHIR Message, switch into a chatty RESTFul API mode, or whatever. The project team is not providing any guidance for each of those more specifically. It is also not setting an expectation that it is relatively easy to go to FHIR APIs, assuming you mean with that "individual resource level API granularity" rather than "message level API coarseness" which are both FHIR APIs. That term is assumed to be fairly limited in scope by most.

Agreed that attempting to switch from a v2 message paradigm into a chatty, resource at a time API approach (RESTful or otherwise) is a bit strange. The intent of a message to begin with is to get all the relevant data across in one go. Clearly you can still use RESTful APIs for that (i.e., a RESTFul API need not be at the granular resource level, although that is what most seem to assume. Writing a Bundle of type message to an API still could use a RESTFul API I guess), but the more fundamental question would be how to have different interactions using non-messaging to enable a full workflow that otherwise would be done using HL7 v2 messaging. That is not part of the v2-FHIR project. Just strictly ensuring that those interested to tap into existing HL7 v2 streams to understand how to map out of that, and in the future how to map into that using existing v2 and FHIR terminology. That could then also be used to help understand implementers figure out how their internal structures now map to FHIR where they are already familiar with v2.

Craig McClendon (Mar 04 2022 at 17:42):

To add to @Cooper Thompson 's comments, repeating segments aren't the only difficult challenge. Handling of not-populated elements, explicit nulls, and repeating segment groups are also challenging.

From the v2 spec:

A field may exist in one of three population states in an HL7 message:
Populated. (Synonyms: valued, non-blank, not blank, not empty.) The sending system sends a value in the field. For example, if a sending system includes medical record number, that would be communicated as |1234567^^^MR^KP-CA|. Note that the field might be populated with a code that means "no information" or "unknown".

Not populated. (Synonyms: unpopulated, not valued, unvalued, blank, empty, not present, missing.) The sending system does not supply a value for the field. The Sender might or might not have a value for the field. The receiving system can make no inference regarding the absence of an element value if there is not a conformance profile governing the implementation. However, if there is a Conformance Message Profile in effect, then special rules apply; see section 2.B, "Conformance Using Message Profiles".

Null. Any existing value for the corresponding data base element in the receiving application should be deleted. This is symbolically communicated as two double-quotes between the delimiters (i.e., |""|).Employing consecutive double quote characters as the only content of a field for other purposes is prohibited.

You essentially need merge logic to update existing resources in place to handle these things. For the case of explicit NULL values, you can't carry that into a blindly transformed FHIR resource - i.e. can't put "" into a dateTime attribute. I suppose you could come up with extensions that carry some of the v2 semantics and tooling that knows how to interpret and merge an intermediary v2-transformed resource into an existing resource on a FHIR server, or perhaps using FHIR patch.

René Spronk (Mar 04 2022 at 18:42):

However, the use of "" in v2 is exceedingly rare, and quite often the fields/components that are populated with ""(in a given context) are relatively well known.

Craig McClendon (Mar 04 2022 at 19:20):

Thanks for the context @René Spronk - it's something I've wondered about as far as how common that may be in the wild. Does that imply that not populating previously sent elements is also rare? For instance sending patient IDs but not full demographics in some messages? Or is that something that would be more dictated by the trigger event?

Cooper Thompson (Mar 04 2022 at 21:23):

We (Epic) do use "" as a deletion indicator in several of our v2 interfaces. I'll admit it isn't universally implemented across all our fields on all message types, but several fields do use it. And since those interfaces are fairly commonly used by many healthcare systems, I think "irregularly used" might be a better description than "exceedingly rare".

René Spronk (Mar 05 2022 at 09:10):

Messaging is 'snapshot' in nature, so a sender sends everything they have/know, time and time and time again. Messaging is also making works case assumptions: the receiver may never have heard from this sender before, nor anything about this patient. The use of "" kind of assumes the receiver already has some data which we want to explicitly delete. So "" is a bit of a strange beast in v2, and is "irregularly used". If I were to create a v2-FHIR transformation I'd be working on the assumption that the use of "" is very low, so PATCH would not be the default way of handling v2-FHIR mappings.

Frank Oemig (Mar 05 2022 at 09:57):

The two double quotes are NOT equal to null. That are two distinct concepts. We are fighting for years to clarify that. Some tables, which are also weird, have values equal to null, or at least similar, like UNK.
The "" are 'delete indicators': the recipient must delete the information he has.
With null values the recipient can keep them.
A null value on V2 is an empty field/component.
The "" are used erroneously when fields are marked as R, and/but no information is available.
That in return indicates, that v2x combines a core framework with aka basic profiles. For V2+ I have tried to separate that, allowing for cleaner specs.
My question here: would that be appreciated or endorsed?

René Spronk (Mar 05 2022 at 12:22):

Sure - in training courses we state that "" means "known to be empty", as in the example: this patient is known not to have any academic titles (thus using "" in the academic title component of a patient name).

|| and ^^ are null flavours (for some reason a normal value [which includes ""] isn't known nor available when sending a message). Other than educating the HL7v2 community I don't see how we could improve the understanding of these two different concepts. Profiling would allow one to state whether null flavours are allowed or not, off the top of my head I don't really see a use case for it in the v2 space however. R means: no null flavours, there has to be a normal value (which could be "" if the value is known to be empty).

Frank Oemig (Mar 05 2022 at 19:18):

R means to get a value which "" is not. If you extend your notion to the maximum that would mean to send only "" which seems weird.
Again, null flavors and delete something are not the same concepts. With "" the recipient has to delete what he has, with a null flavor he can keep it.

René Spronk (Mar 06 2022 at 09:14):

To qoute v2 (which does indeed indicate my wording to be erroneous, 2.7 ch. 2.5.3.0):

A field may exist in one of three population states in an HL7 message:

1. Populated. (Synonyms: valued, non-blank, not blank, not empty.) The sending system sends a value in the field. For example, if a sending system includes medical record number, that would be communicated as |1234567^^^MR^KP-CA|. Note that the field might be populated with a code that means "no information" or "unknown".

2. Not populated. (Synonyms: unpopulated, not valued, unvalued, blank, empty, not present, missing.) The sending system does not supply a value for the field. The Sender might or might not have a value for the field. The receiving system can make no inference regarding the absence of an element value if there is not a conformance profile governing the implementation. However, if there is a Conformance Message Profile in effect, then special rules apply; see section 2.B, "Conformance Using Message Profiles".

3. Null. Any existing value for the corresponding data base element in the receiving application should be deleted. This is symbolically communicated as two double-quotes between the delimiters (i.e., |""|).Employing consecutive double quote characters as the only content of a field for other purposes is prohibited.

René Spronk (Mar 06 2022 at 09:30):

In my example of a persons academic titles: sending ^^ would indicate not knowing if a person has academic titles. "" indicates (as a creator of a message) that I'm aware the person has no academic titles (and if you as the receiver have academic titles in your database, you'd better delete them). That use is somewhat wider than the more limited definition:

Given the definition of "" one could state that "" shall only be used by a sender if they know they've sent erroneous information in the past which they want to explicitly delete. That's a fairly narrow use case, which calls for a lot of complexity on the sender's side. 'To null' in the sense of 'canceling out' some earlier value.

Going back to the formal definition: Populated includes the notion of what in v3 we'd call nullFlavours. The Not Populated option would also be nullFlavours. The only option that doesn't allow nullFlavours is .. Null.

Some updating of the wording may be in order, nit to redefine the meaning, but to align it more with terminology used in subsequent HL7 standards.

Cooper Thompson (Mar 07 2022 at 15:23):

We can continue down the rabbit hole of deletion indicators and null flavors, but what I'm really interested in is whether anyone (implementer) is planning to handle this in a v2 to FHIR product. I'm still really curious what happens when we get the v2 field to FHIR property mapping done. Will that actually be usable? Only usable when doing v2 message -> FHIR messages? Or is anyone realistically thinking they'll be doing v2 messages to REST API?

René Spronk (Mar 07 2022 at 16:48):

It will be usable as a reference mapping, a starting point for defining ones own contextual mapping. It doesn't help when it comes to all sorts of issues one will have to deal with when going from messaging to REST ("" just being a very tiny example of this). Which people are already doing, given the plethora of v2 stuff out there this is probably a must for any large scale project.

Cooper Thompson (Mar 07 2022 at 18:52):

I am really interested in knowing what people are actually doing this. I've heard of folks trying to do this for the various cloud data warehouse imports, but I haven't heard of those projects going well.

Frank Oemig (Mar 09 2022 at 07:11):

We have to think about the mechanics, how it should be, and not what people have done to solve an issue. There are a lot of quick fixes - for good reasons, but those are not helpful in the end.
And one problem are the statements if early days that use improper technical terminology.
V2 has no notion of null flavors. We have some code in tables for that purpose. I know the sentences in the standard, they are misleading. And trying to fix that failed for years...

René Spronk (Mar 09 2022 at 07:33):

Luckily at least FHIR uses 'dataAbsentReason' instead of nullFlavour, this helps to some degree to avoid the confusion. Doesn't help in the case of v2, though. But we can't retrofit a legacy standard..

Andrea Pitkus, PhD, MLS(ASCP)CM, CSM (Mar 23 2022 at 15:08):

From a content perspective, I can see this being a bit issue in cancer and pathology reporting as there are many "flavors of null" in the CAP electronic Cancer Checklists (CAP eCC) content. For example if a field is blank, does it mean it doesn't apply, wasn't seen in the specimen, or did the pathologist skip the question/miss answering, etc.? This assumes the data are messaged discretely. Some may be captured discretely in synoptic reports, but it may not be sent discretely and isn't encoded in a number of cases (although some fields are). Work is occurring in this area....

Last updated: Apr 12 2022 at 19:14 UTC