Stream: implementers
Topic: Conditional Reference
Chris Grenz (Apr 07 2016 at 01:02):
@Simone Heckmann Looking at the proposed connectathon track for conditional references. This is a HUGE need - thank you for taking it on!
Chris Grenz (Apr 07 2016 at 01:04):
I have a question about scenario step 3 - what if the client knows that the referenced item will arrive out of order? There are times, esp. in emergent scenarios, where registration data may lag the order feed for instance. We would like to be able to save the order, retaining the conditional linkage until the referenced data arrives.
Chris Grenz (Apr 07 2016 at 01:05):
However, this: a) causes the server to have to re-check for the conditional linkage later, and b) introduces the possibility of undetectable (to the client) mis-matches.
Chris Grenz (Apr 07 2016 at 01:07):
Right now, we're using contained resources to handle this scenario - we create a contained resource with only what we know about the referenced resource. The server then (preferably on write, but possibly on read for late arriving) resolves the match. If the match can't be resolved, the contained resource remains and the reference remains internal.
Chris Grenz (Apr 07 2016 at 01:08):
Does anyone see pros/cons to either approach? (as described vs. http://wiki.hl7.org/index.php?title=201605_Conditional_Reference_Connectathon_Proposal)
Chris Grenz (Apr 07 2016 at 01:08):
@Grahame Grieve @Ewout Kramer @Lloyd McKenzie @Josh Mandel ??
Chris Grenz (Apr 07 2016 at 01:18):
Talking to myself a bit...contained allows the server more latitude to create a meaningful resource (conditional create) if that fits the server's use case better (contained could contain some arbitrarily large amount of data about the referenced resource). However, there would need to be some directive by the client that the intent is to match/convert the contained resource.
Chris Grenz (Apr 07 2016 at 01:19):
Conditional reference is more explicit and could still serve the late-arriving use case if the conditional reference was allowed to remain in the resource after POST/PUT.
Lloyd McKenzie (Apr 07 2016 at 03:56):
Technically if the inbound instance has a contained resource, the server would have to do an "update" such that the original version (with any signature) was retained. I'm not a fan of storing a conditional reference.
Simone Heckmann (Apr 07 2016 at 09:17):
What you could do is to pack both the referencing and the referenced resource into a transaction, using temp uuids to point from the referencing to the referenced resource.
Simone Heckmann (Apr 07 2016 at 09:18):
The server sill then replace the temporal ID with the actual ID ans update the reference
Simone Heckmann (Apr 07 2016 at 09:19):
If - at th point of submission - you have no information about the referenced resource, you can submit whatever you have (at least the business identifier) as a conditional create.
Simone Heckmann (Apr 07 2016 at 09:21):
That way, the server sill create your resource, if it doesn't exist or pick an already existing resource with the same identifier and reference to that resource instead without updating it. (Provided, your condition is the matching of the business identifier)
Simone Heckmann (Apr 07 2016 at 09:22):
Once the actual data arrives, you can update the resource by using a conditional update. (Once again by matching the identifier)
Simone Heckmann (Apr 07 2016 at 09:24):
This way you can have a unidirectional data stream to the server without ever having to query for IDs (which is pretty much what sending V2 streams to a fhir server is all about)
Simone Heckmann (Apr 07 2016 at 09:24):
I think the conditional reference is only a good choice if you can savely assume that the referenced resource already exists on the target server.
Simone Heckmann (Apr 07 2016 at 09:34):
e.g. every Hospital in a region/country is commonly known by a unique national identifier. In this scenario I could simply always reference to a Organisation with that identifier, without having to look up the different IDs of the different servers you're talking to. In such scenarios, you'd explicitly WANT your submission to fail, because a no match by identifier means that the referenced hospital is not participating in your network.
Simone Heckmann (Apr 07 2016 at 09:38):
or: you have some sort of automated workflow where a barcode sticker (case number) is read from a sample, a test is performed and the result is transmitted back to the HIS. In that scenario you have a strong reason to belive that the case number is known to the HIS because it printed the barcode sticker in the first place.
Kevin Mayfield (Apr 07 2016 at 10:06):
I like the idea. I currently use contained resources to just pass around the ESB/Hospital indentifiers and so in most cases they just contain identifier.system and identifier.value. I don't use the contained resources to create resources (within the hospital/ESB). It's a bit tedious to build and process. If a shorthand reference was available I would just use that.
James Agnew (Apr 07 2016 at 12:09):
+1 to Simone's recommendation
Simone Heckmann (Apr 07 2016 at 13:15):
FYI: James has that already implemented conditional references in HAPI.
Simone Heckmann (Apr 07 2016 at 13:16):
The Transaction/Conditional Update/Conditional Create- Workflow I described above has been successfully tested at the Connectathon in Atlanta.
Simone Heckmann (Apr 07 2016 at 13:16):
Not sure about the other servers, though.
Chris Grenz (Apr 07 2016 at 14:09):
I actually like the conditional reference - it's simple. But take a moment to read the description of contained resources (and references to them): https://www.hl7.org/fhir/references.html#contained They exist for a flavor of this scenario - a client knows only a little about the referenced resource, so it creates a contained. The intent (usually) is *not* that the contained remain forever, but rather that the correct match and reference be made. Contained is a concession to the reality of imperfect information.
Chris Grenz (Apr 07 2016 at 14:12):
The description of contained would lead me to believe that we should be defining some directives on what a server should do with these things - not defining matching logic necessarily, but at least a way for the client to say "please try and match this partial data to something you know about".
Chris Grenz (Apr 07 2016 at 14:15):
@Simone Heckmann A conditional create would be fine IF the server's profile/logic allows. It's certainly plausible though that a Patient record with only an identifier would be considered inadequate to create a resource (as the contained explanation relays).
Chris Grenz (Apr 07 2016 at 14:29):
It may be a situation where both (all 3?) approaches are appropriate as long as there is a clear explanation of the utility and appropriateness of each.
Lloyd McKenzie (Apr 07 2016 at 18:11):
Contained resources generally would not mean that the server can turn them into non-contained references. Doing so would certainly break signatures, so it would need to be treated as an update that still retained the existence of the original version if signatures were at all important.
Chris Grenz (Apr 07 2016 at 18:31):
Lloyd - agree that the conversion would be an update, but it seems to me that this should be able to be requested by the client, especially in the messaging world. I can't think of a scenario that would prefer a partial contained to a real reference. The contained could even be retained for posterity, but the local references should be updated if desired.
Chris Grenz (Apr 07 2016 at 18:31):
Also, the signature of a resource with a conditional reference converted to a "real" reference would also be altered.
Lloyd McKenzie (Apr 07 2016 at 21:04):
I don't think you can assume a server will ever do post-processing. So a conditional create followed by a subsequent conditional create or conditional update is what would best give the client control over what's going on.
Chris Grenz (Apr 07 2016 at 21:05):
Isn't resolving a conditional reference "post-processsing" - a subsequent GET will not return what was just PUT.
Lloyd McKenzie (Apr 07 2016 at 21:07):
Resolving a conditional reference is post processing that's done at the time of the create/update. If it fails, the create/update fails. I'm talking about deferred post-processing (as would be required for a system to update a resource with a "contained" resource to instead point to a non-contained resource.
Chris Grenz (Apr 07 2016 at 21:12):
The contained could be resolved at write time or (if matching fails) retained as-is as a matter of standard operation. Matching/resolution after that point would be a server-specific choice.
Lloyd McKenzie (Apr 07 2016 at 21:47):
I'm not thrilled with it being retained as-is, nor with any implicit assumption that servers "should" be responsible for making such updates. If we're doing REST, I think it's cleaner for things to be client-driven
Chris Grenz (Apr 07 2016 at 21:48):
Isn't that the current operation of contained resources? I don't know enough to create a resource, so I'll put in whatever I have. If that's not enough to match, it would be natural to leave it alone, no?
Chris Grenz (Apr 07 2016 at 21:50):
Any "post-processing" would be up to the implementer. You're right that a ReST client shouldn't assume.
Chris Grenz (Apr 07 2016 at 21:50):
Including assuming that other actors (e.g. a post-processor) won't act on a resource after their operation is complete.
Lloyd McKenzie (Apr 07 2016 at 22:00):
When you create a contained resource, you expect it to remain contained until/unless a client updates it. Which means the client has to remember to update it (which is unlikely).
Chris Grenz (Apr 07 2016 at 22:02):
I disagree. There's nothing that would prevent the server from implementing (according to its own needs) a process for this, effectively acting as an internal "re-indexing" client. I agree that the standard shouldn't mandate this - it should define only what happens during the client interaction.
Chris Grenz (Apr 07 2016 at 22:02):
A client should not assume that it's the only client.
Paul Knapp (Apr 08 2016 at 11:47):
@Kevin: We have the same situation in the financtial domain where the identifier for a think is known but the id of a content model (resource) is not - so were also createing contained resources sometimes only to contain the identifier. We have recently change those elements for which this is the case from resouce(x) to Identifier|resource(x) - see CLaim and the examples SOA Dental Claim, which uses contained resources, and SOA Dental Claim with identifiers which doesn't.
Kevin Mayfield (Apr 08 2016 at 12:18):
Do you have a url for those examples? I'm wondering if contained resources has too much coupling for the client. National identifiers (UK NHS Number or even EU EHIC Card number) is enough information to reference a patient resource and exchange information. Thinking of an Observation app which for example conforms to a pan EU Observation profile, it's unlikely to support all the Patient profiles in the EU - identifiers yes. It should be the servers job to get the Patient (with the correct profile) if it needs to, in many cases the server will own the master Patient resource.
Kevin Mayfield (Apr 09 2016 at 07:42):
Looking at the proposal I've seen references such as <reference value="Patient?identifier=9876512345"/> would references such as <reference value="Patient?identifier=urn:fhir.nhs.uk:id/NHSNumber|9876512345"/> work? i.e. also include the system
Kevin Mayfield (Apr 09 2016 at 07:49):
@James Agnew I notice this is already supported in HAPI, is that 1.4? I have some code which shows the number of lookups we're doing when posting a DocumentReference - it would really speed up the processing if it does (6 calls downto 2). Apache Camel route is here https://github.com/KevinMayfield/Jorvik/blob/master/UKFHIR-HAPI/src/main/java/uk/co/mayfieldis/camelRoute/UKFhirAPI/UKFHIRCamelRoute.java
Josh Mandel (Apr 10 2016 at 16:54):
As much as I dislike "contained" resources, one nice thing about that approach is that it avoids the potential to create invalid references on the server -- e.g. if a client sends data with a "conditional reference" and the server doesn't know how to process this, and so the client's submission just lands directly in the server's database. You'd wind up seeing data with references like:
<reference value="Patient?identifier=123"/>
Since a reference is just a string, this is pretty likely in some server designs. And following the reference would resolve a bundle instead of a resource, which would be surprising/unpleasant for clients.
Paul Knapp (Apr 10 2016 at 18:40):
@Kevin: In FM resources to support situations such as national patient identifiers, national practitioner identifiers etc we put in the choice of a resource reference or an identifier so you could have patient system = "http://nationalsystem.com/patientidetifier" value = "AB123456". We are starting to hear pushback that we should instead create a separate resource, included the reference to that and either make you come read that resource too or we can include the original resource and all of the other created resource instances in a larger structure (bundle, message, document, etc). Not very efficient, nor usefull especially when you aren't giveing the end party a patient to add or update on their system, we are just tryin to say 'it's patient/provider/whatever number 12345'. Doable but is the pain worth the gain?
The NHS or EHIC card number is an identifier not a resource.id and FM has been trying to discourage confusing them and avoiding unnecessary overhead. All the providers who service that patient would give you completely different resource.ids to locally created resources which contain the same identifier. It seemed to us to be more efficient as you indicated to just give the identifier as that is what you need to know.
We were trying to be explicit by allowing one to declare and provide the identifier. If that's not allowed the above seems more obscure but still closer to the same efficientcy, my example could be: <reference value=Patient?identifier?=http://nationalsystem.com/patientidentifier|AB123456"/>.
Lloyd McKenzie (Apr 10 2016 at 20:31):
The intention with the reference approach would be that the submission would fail unless the reference could be resolved to an existing resource though.
Grahame Grieve (Apr 11 2016 at 04:44):
catching up on this thread: contained resources were created for the case where you don't have enough information to identify the resource. Which is the exact opposite of when you have an identifier that can be resolved. So contained resources are a different problem to the one being discussed here
Paul Knapp (Apr 11 2016 at 17:43):
Ok then the pattern which has emerged is that if you can uniquely identify the thing you should create a resource and reference that not create a contained resource.
So that would eliminate having bodySite as a choice of the code and the resource as the code would uniquely identify the resource.
Grahame Grieve (Apr 11 2016 at 18:14):
where do we have bodysite as a choice of code and resource?
Paul Knapp (Apr 11 2016 at 18:17):
We do in claim but I got that from somewhere else, we just need the code, - I'll look.
Andy Stechishin (Apr 11 2016 at 18:18):
Observation uses bodySite[x] codeableConcept|Reference(BodySite)
Grahame Grieve (Apr 11 2016 at 18:26):
not in DSTU 2 or the current version
Paul Knapp (Apr 11 2016 at 18:39):
DeviceuseRequest, DeviceUseStatement, MedicationAdministration for a start and MedicationAdministration uses the same pattern to Medication in current build.
Grahame Grieve (Apr 11 2016 at 18:43):
ok tahnks
Eric Haas (Apr 11 2016 at 19:43):
I think those have open trackers to change or should have them
Paul Knapp (Apr 11 2016 at 19:46):
And Careplan and Procedure and SupplyRequest use the same pattern too.
Paul Knapp (Apr 11 2016 at 19:56):
And Specimen and maybe ModuleDefinition.
Eric Haas (Apr 11 2016 at 20:59):
@Paul Knapp not in dstu2 and current build all PC and OO resource only have codeableconcept for Bodysite
Lloyd McKenzie (Apr 12 2016 at 03:24):
For BodySite, it was decided that the reference to the resource was extension space. However, for referencing Medication, both codes and ability to point to a full-blown resource (for things like compounds) are both in the 80%. Whether the code is sufficient to be "identifiable" varies - sometimes yes, sometimes no.
Paul Knapp (Apr 12 2016 at 05:48):
@Eric Prud'hommeaux: Specimen appears to use the pattern in question just not for BodySite.
@Lloyd McKenzie: Was anyone told it was extension space? By that do you mean that resources which use the code need to use an extension rather than a choice to use a reference? What medictions can you specifiy by code which you could not put in a Medication resource?
Grahame Grieve (Apr 12 2016 at 06:03):
It's not that you can't specify them in a medication resource, it's just that many applications simply use a code, and the committee came to the conclusion that it was appropriate to allow either a code or a reference to a medication resource - and for many medication terminologies, we'll be defining formal equivalence between their codes and an equivalent medication resource.
Grahame Grieve (Apr 12 2016 at 06:04):
I don't know what you mean by 'told it was extension space' - the committees are presently making the determination of what is common or not
James Agnew (Apr 12 2016 at 14:14):
@Kevin Mayfield Re Conditional reference in HAPI, it's only supported as of 1.5 and FYI it needs to be explicitly enabled in the JpaConfig since it's only an experiment for now. You can try it out with the current snapshots, and we'll be launching 1.5 this week most likely.
Kevin Mayfield (Apr 12 2016 at 15:13):
Thanks @James Agnew WIll take a look
Lloyd McKenzie (Apr 12 2016 at 15:32):
@Paul Knapp The extension is defined in the extensions associated with the BodySite resource. Not sure what you mean about "telling". If implementers find they need something that isn't in core, the first thing they should do is look to see if there's a standard extension. Some of the medications are "abstract" - e.g. "Aspirin" or might even just have a name (CodeableConcept.text) without a code at all. In those cases, a stand-alone Medication couldn't be used, though a contained resource could be. But it was decided that forcing a contained Medication when the use-case of having a bare code was super common (probably 70%+ of instances) was inappropriate. In the case of Patient, the considerations are a bit different. We don't have Compartments for Medication. And querying for orders or other resources by Medication or joining resources in queries by Medication is quite uncommon, while querying and joining by Patient happens all the time. So the considerations are different. That doesn't mean we can't allow the pattern, but if the pattern gets used, it should be used everywhere and we need to think carefully about the additional ramifications.
Josh Mandel (Apr 12 2016 at 20:17):
@Lloyd McKenzie @Grahame Grieve my concern with "conditional references" in POSTed resources is that suddenly it's not okay for a server to just persist a reference; it needs to detect when a reference may be conditional, and then behave appropriately. But previously we've said that a reference can be an arbitrary URL. We'd need to document expectations carefully.
Grahame Grieve (Apr 12 2016 at 20:23):
well, many servers already enforce that the references are valid. or index according to them (you pretty much have to do that). so merely making a server inspect references is not that radical. Agree that we'd have to be very clear about this though
Simone Heckmann (Apr 13 2016 at 08:26):
There's also the (still somewhat undecided) idea out there to only support conditional references in Transactions. In that case, the pure REST server could still behave as Josh described. In the Transation, the server has to inspect references anyway, since we already defined the expectation that the server must replace uuids. So nothing new here.
Last updated: Apr 12 2022 at 19:14 UTC
