Stream: implementers
Topic: supporting multiple fhir versions
Kevin Olbrich (Mar 08 2017 at 13:42):
In some use cases of FHIR servers there may be compelling reasons (graceful version upgrades, disparate versions in connected systems, etc) to build the server so that it can accept/emit more than one version of FHIR, but there does not seem to be any mechanism for negotiating which version to use. Does anyone have any thoughts about how best to handle these scenarios?
Lloyd McKenzie (Mar 08 2017 at 16:30):
While we're still in the STU phase, this is difficult because there's no backward/forward compatibility between versions. The typical approach is to create multiple server endpoints - one for each supported version. The endpoints then handle translation to and from the internal interface layer (whether that's a single FHIR version or something proprietary). As we move to normative, it should be possible for different versions to all be posted to the same endpoint. The server is tuned to the most recent version it supports and ignores data that's based on future versions (as the rare data element added to core in the future is guaranteed to be ignorable)
Kevin Olbrich (Mar 08 2017 at 16:59):
I suspect that approach is going to make deployment, maintenance, and upgrading of FHIR applications and servers quite painful. I'm already encountering this pain during the development of new applications. Here are a couple of articles describing a couple of ways of handling this... https://www.narwhl.com/2015/03/the-ultimate-solution-to-versioning-rest-apis-content-negotiation/, https://www.troyhunt.com/your-api-versioning-is-wrong-which-is/.
Lloyd McKenzie (Mar 08 2017 at 18:04):
Which part of the approach are you referring to? The part prior to normative or the part after normative? There's no question that the phase prior to normative is painful, but it's a pain implementers knowingly take on in choosing to implement a spec that's not yet locked down. And we're not willing to incur the cost of complexity/baggage/brokenness that comes from locking down the spec or maintaining backwards compatibility with specifications that haven't yet been well beaten on by the community. If you're seeing pain in the approach for post-normative, please describe your concerns further
Michael Lawley (Mar 13 2017 at 21:25):
I've railed about this before. The current practice of having FHIR version numbers for the spec while it's changing that are indistinguishable from version numbers that represent frozen forms is really frustrating.
Grahame Grieve (Mar 13 2017 at 22:44):
why is it frustrating?
Michael Lawley (Mar 13 2017 at 22:45):
Because you can never know, just by looking at the version string, whether the thing it references is fixed or (potentially) moving
Grahame Grieve (Mar 13 2017 at 22:45):
there's only one vrsion that's moving. The current version
Michael Lawley (Mar 13 2017 at 22:46):
But how do you know what that is (just by looking at the version string)?
And then one day, that very same string is no longer "current", so it's status changes.
Grahame Grieve (Mar 13 2017 at 22:47):
you can't, just by looking at the version string, but why would it matter?
Michael Lawley (Mar 13 2017 at 22:52):
If I (my client software) looks at a capability statement and sees 1.9.0, then today it's fair to accept that it is current and there may be variation in behaviour (eg the public HAPI server uses content rather than content-mode for CodeSystem search), but at some point 1.9.0 will no longer be 'current' so should be fixed, but I can't rely on a server that reports 1.9.0 actually implementing the "latest 1.9.0", just "one of" the 1.9.0s
Michael Lawley (Mar 13 2017 at 22:54):
If there was a policy of odd vs even minor versions (even being fixed, odd moving), or x.y.z-SNAPSHOT or x.y.z.commit then a client could be confident when a reported version actually referred to the frozen instance of the spec.
nicola (RIO/SS) (Mar 13 2017 at 22:55):
Let me put it again - https://www.youtube.com/watch?v=oyLBGkS5ICk, semantic vesioning is antipattern :(
Michael Lawley (Mar 13 2017 at 23:00):
1hr 15m is a long time to invest -- I can't find the slides; is there a (longer) TLDR :)
nicola (RIO/SS) (Mar 13 2017 at 23:01):
here is some abstract - http://blog.ezyang.com/2016/12/thoughts-about-spec-ulation-rich-hickey/
Grahame Grieve (Mar 13 2017 at 23:01):
That's an an-pattern of it's own
nicola (RIO/SS) (Mar 13 2017 at 23:03):
I've reported how we updated 3 major versions of clojure and all libraries on some project and all tests was passed - it's true! Java backward compatability! HTML
nicola (RIO/SS) (Mar 13 2017 at 23:04):
May be it looks little bit messy, but life will be much more simple! Every expirienced engeneer knows how to refactor/change in a safe way.
Michael Lawley (Mar 13 2017 at 23:05):
None of this really helps my problem though. When I see a server reporting fhir version "X", I can't be confident that it actually implements "the latest X"
nicola (RIO/SS) (Mar 13 2017 at 23:05):
We do not need version at all :)
nicola (RIO/SS) (Mar 13 2017 at 23:05):
feature detection - like in CSS :)
nicola (RIO/SS) (Mar 13 2017 at 23:07):
If you need name from Patient - you just check - there is Patient.name - that's all
nicola (RIO/SS) (Mar 13 2017 at 23:08):
This will open ability for standard to evlolve continuously (every week or even day) - this is like agile in process, continuous delivery in operations
nicola (RIO/SS) (Mar 13 2017 at 23:10):
If everybody agree on some element - it could be published and used next week
nicola (RIO/SS) (Mar 13 2017 at 23:11):
That what's happening with software last 20 years - spinning up release cycles to exremes.
Michael Lawley (Mar 13 2017 at 23:11):
If you need to search for CodeSystems based on content=complete, then you need to first retrieve /metadata (sometimes >500k in my experience, even though much smaller than in DSTU2 with CodeSystem info)
nicola (RIO/SS) (Mar 13 2017 at 23:13):
versions could be like git commit - SHA from metadata
Michael Lawley (Mar 13 2017 at 23:14):
but...if you have a client-side mechanism that makes v ersion numbers immaterial for you, that's fine. It shouldn't be an argument against fixing something that's inherently brittle
nicola (RIO/SS) (Mar 13 2017 at 23:15):
Backward compatabilyt will work with static typed clients too :)
nicola (RIO/SS) (Mar 13 2017 at 23:16):
You just have to detect version >= i need
nicola (RIO/SS) (Mar 13 2017 at 23:30):
intensive vs extensive dichotomy
Kevin Olbrich (Mar 14 2017 at 13:09):
@Michael Lawley would something like an 'unstable' boolean flag on the capabilitystatement be sufficient for your purposes or are you in need or something more fine grained?
Kevin Olbrich (Mar 14 2017 at 13:12):
@nicola (RIO) I like the idea of feature detection as a way of making versioning a moot point.
Michael Lawley (Mar 14 2017 at 13:25):
@Kevin Olbrich Adding an extra property to the Capability statement seems like overkill when just using 1.9.0.unstable (or an existing convention with well-known ordering rules like 1.9.0-SNAPSHOT) would do just as well.
nicola (RIO/SS) (Mar 14 2017 at 13:28):
@Kevin Olbrich I think, for global FHIR application - feature detection & backward compatability is a right trade-off - i'm pretty sure, if FHIR will get global adoption - many versions will coexist - why not make it legal and reduce pain of migrations and interactions. This just requires following simple rules in FHIR design and evolution - don't change or remove, only add.
Michael Lawley (Mar 14 2017 at 13:29):
@nicola (RIO) I take your point, and that's pretty much how we operate with our Snapper client, but I see these version strings also as convenient labelling of a collection of features (also, not everything that's important is discoverable through the Capability statement).
Even Git has tags, etc and not just SHAs
nicola (RIO/SS) (Mar 14 2017 at 13:31):
@Michael Lawley I agree
Kevin Olbrich (Mar 14 2017 at 13:31):
@nicola (RIO) Isn't that principle of don't change or remove, only add more or less in line with what one would expect with a normative release?
Kevin Olbrich (Mar 14 2017 at 13:32):
I would also love to see better use of tagging for versions. It would be great if I could go to the github project and get a list of releases and tarballs of the artifacts related to that release at a nice, predictable url.
Michael Lawley (Mar 14 2017 at 13:32):
One more thing, how does a DSTU2 (for example) client tell a not-yet STU3 server that it expects to get a DSTU2 ConceptMap without the group property? The content negotiation thing has to work in both directions
nicola (RIO/SS) (Mar 14 2017 at 13:34):
@Kevin Olbrich I hope. But it's better to develop culture of 'non-breaking' changes from the early time!
nicola (RIO/SS) (Mar 14 2017 at 13:35):
@Michael Lawley i think, it shoud not tell - don't give me something, but - give me something, if you are able - open world principle
Kevin Olbrich (Mar 14 2017 at 13:35):
In general I agree with that, but it strikes me that there has been a lot of thrash in the early days of FHIR and trying to keeps all of that around indefinitely is not practical.
Kevin Olbrich (Mar 14 2017 at 13:37):
If a client could inspect the schema of the resources that a server provided, it could determine if a particular attribute existed and was the right format. Seems like that might add a lot of overhead on the communications.
nicola (RIO/SS) (Mar 14 2017 at 13:37):
That's trade-off, a lot of obsolete code in java - they do compaction, but after several years after deprication.
Michael Lawley (Mar 14 2017 at 13:44):
indeed - by "tell" I meant "express" / "request". With stable, predictable version strings the Accept header could be used for this (give me a application/fhir+json;fhirVersion=1.4.0 rendering of the resource)
nicola (RIO/SS) (Mar 14 2017 at 13:45):
Or application/fhir+json;fhirVersion=>1.4.0 :)
Kevin Olbrich (Mar 14 2017 at 13:46):
yeah, I've been considering using the content type stuff so that a single server could handle multiple versions of fhir.
nicola (RIO/SS) (Mar 14 2017 at 13:48):
Good idea, we are working now on multi-version server (but for different tenants) - i will try to serve old versions api from newer endpoints!
Kevin Olbrich (Mar 14 2017 at 13:49):
what language are you using to implement that?
nicola (RIO/SS) (Mar 14 2017 at 13:52):
@Kevin Olbrich the best one - clojure
Michael Lawley (Mar 14 2017 at 13:54):
Sadly, the > is an invalid character in a token as per the Accept Header syntax. But, I dont think that's needed. A client will (should) ignore any extraneous stuff, sp the version string here is about identifying (too broadly, I know) what the client desires. So if stuff is only ever added in later versions, then =>1.4.0 is semantically equivalent to =1.4.0 When it isn't, then you have breakage, and =>1.4.0wouldn't suit the client anyway
nicola (RIO/SS) (Mar 14 2017 at 13:55):
I converted all versions metadata (structure def, search params etc) into internal unified format - so, i think, to implement multi-version server will not be big challenge (may be only some corner cases).
nicola (RIO/SS) (Mar 14 2017 at 13:56):
Looks like this is possible - convert versions on fly - i will report results.
nicola (RIO/SS) (Mar 14 2017 at 13:57):
@Michael Lawley good point
Kevin Olbrich (Mar 14 2017 at 13:57):
I am also thinking that the CapabilityStatement should allow for (1..*) fhirVersion elements...
Kevin Olbrich (Mar 14 2017 at 13:58):
but that might cause some problems with older clients in particular.
Kevin Olbrich (Mar 14 2017 at 14:00):
I suppose we could add a new attribute 'standardVersion' that would be allow multiple versions to be reported.
Mario Hyland (Mar 14 2017 at 14:59):
For those FHIR Developers/Implementers testing with Touchstone - they are able to see and test either client or server against various FHIR published versions. This will afford the community an opportunity to see how backwards / future proofing a solution is. As we test FHIR 1.8.0 vs 1.9.0 we can see how much the spec has changed. I agree with the multiple end points supporting a capability statement based query by a client to server - prior to any information exchange occuring. I think the client should performance some due dilengence on the server. Not sure how the server will do the same of the client :( Interesting to see some IG and FHIR approaches on one hand reference DSTU 2 and call for resources from STU3 (not sure how that is going to work out).
Grahame Grieve (Mar 14 2017 at 19:21):
several comments:
- so it would be good to have a task for documenting a version parameter on the accept header
- versioning will work exactly as Nicola would like once we go normative, and that's planned for the next release for the core (not sure what that is yet)
- already, a client can do everything by introspection, rather than by versioning. It's just hard work
- our intent has always been that there will be no need to do version negotiation
Mario Hyland (Mar 15 2017 at 02:14):
Ok, I get some of this:
- "our intent" ... no need to do version negotiation - based on what we have seen, even with updated Provider Directory capabilities and published end-points which include version specific details. Both FHIR Client and Server (including FHIR Services) may want to take an additional step as a percaution. Whether introspecting a header, or interrogating a capability statement. Or am I to read your comment as you are stating we should warn folks off from this approach?
Lloyd McKenzie (Mar 15 2017 at 02:38):
@Mario Hyland There's a difference between the behavior we have right now in STU where there's no backward compatibility requirement and we can (and do) make breaking changes between versions and what we'll eventually have when things are locked down. But once we're at normative, systems shouldn't care about version - they just check whether the elements they care about are present and ignore what they don't expect. (And are safe when doing so.)
Kevin Olbrich (Mar 15 2017 at 13:35):
@Grahame Grieve How do you envision the 'introspection' approach working? Say if I want to know if a server can handle a minimum set of resources and attributes in order to know if it is capable of handling a particular integration, how would I figure that out? The resources I can get from the CapabilityStatement, but if there was an attribute that is present on a 1.8.0 server that is not on 1.4.0, how would I figure that out?
Lloyd McKenzie (Mar 15 2017 at 14:47):
Supporting a given version doesn't mean you'll support all of the core elements from that version. If your question is "will the server support the following 5 attributes?", the only way to know is to retrieve the CapabilityStatement and trace through to the StructureDefinition profiles for each resource and see what elements the server supports.
Kevin Olbrich (Mar 15 2017 at 14:53):
@Lloyd McKenzie Yeah, I see that now. Thanks.
Mario Hyland (Mar 22 2017 at 21:13):
FHIR Web Site Version page - http://hl7.org/fhir/directory.html - FHIR Version 3 Official should also be BOLD - similar to DSTU 2 and DSTU 1
Grahame Grieve (Mar 22 2017 at 21:27):
ok. will be updated soon (back of a queue of 500k files being uploaded)
Chris Grenz (May 17 2017 at 22:42):
After the discussion in Madrid, I wrote an article on this topic: https://www.linkedin.com/pulse/fhir-nuts-bolts-versioning-chris-grenz
Chris Grenz (May 17 2017 at 22:43):
Or at least a closely related topic!
Last updated: Apr 12 2022 at 19:14 UTC
