Stream: R4A/B/R5 Discussion
Topic: Managing change
Lloyd McKenzie (Mar 05 2021 at 23:22):
Continuing from sidebar discussion in https://chat.fhir.org/#narrow/stream/250700-R4A.2FB.2FR5-Discussion/topic/Version
@Gino Canessa, by your definition, safe to use = normative and maybe FMM 4 & 5. But unlikely != won't, so it's a tough call. We want people to use resources regardless of whether they're 'safe' by your definition - because they'll never get their without implementation.
In the case of subscription, you can get real-world experience of much of the approach via the pre-adoption IG, though that too will take time to get into production. The reality is that every resource has its own adoption trajectory and influences on stability, but real solutions are cobbled together with lots of arbitrary collections of them. Some implementers are ready for change long before others. If we had put out R5 at the beginning of this January (our original plan), would the implementer community have been willing to do anything with it? (My guess is 'no' given the number of regulations tied to R4 and the work needed to comply with them over the next couple of years.)
My leaning is that we partition FHIR into two 'logical' pieces. The first part contains all of the normative and high-stability resources. We plan to update those on a roughly 3-year cycle (with the possibility of technical corrections and non-substantive updates to examples, clarifying wording, etc. if we can figure out how to make those not eat a month of Grahame's time). The other part would be all of the low-maturity resources and we plan to produce at least one breaking release of those in between the 'big' releases. That gives us a 'short' iteration path for the immature stuff (while still being long enough for us to manage as an organization), while letting people easily comply with two releases if they don't use any of the immature resources. Pretty much everything in US Core, even if it isn't normative, would fall into the 'mature' state. We'd have to make the call on where subscription should fit. As an infrastructure resource, a lot of people are going to want it in the 'stable' category, but I accept that its maturity level isn't too high from a real-world experience perspective.
Gino Canessa (Mar 06 2021 at 00:20):
I'm trying to move the discussion forward by standing up two proposals around the same issue - that while we want parts of the standard to be stable for longer periods of time, slowing the unified releases impedes the progress of the rest of the standard. In one version, we make an explicit cut based on the 'stability' of resources (e.g., 'core' and 'extended', 'stable' and 'experimental', etc.). In the other version, we make an implicit cut based on FMM.
Both approaches have pros and cons, and that addresses just one piece of the puzzle. For example, I would argue for the 'experimental' builds (via whichever approach) to be tied to the Connectathon cycle. That way, we minimize the lag in making changes to testing them to publishing them. I'm working on the assumption that WGs are both responsible enough and the best judges of when a breaking change is actually warranted.
Whatever is adopted, I am firmly against divergence. I can attest that it is both painful and error-prone applying documentation changes to R5 and an IG at the same time. If there are multiple, different source trees for the core specs this would quickly be unmanageable.
As to real-world experience with subscriptions based on the backport... yes and no. There are areas that were deliberately left out of the backport because they are too complicated (e.g., there is no equivalent to SubscriptionTopic) or can be done differently with the R4 version of subscriptions (e.g., on-demand topics). Without testing the actual R5 models in production, I can't support moving them forward.
And to be blunt, I'm worried that the backport will cause issues as we move towards R5. For example, we are using a Parameters instead of a SubscriptionStatus for meta-information in the bundle, since it doesn't exist in R4. In 202x, when people have years of implementations with Parameters, I'm guessing we will have arguments against introducing the 'breaking change' that is a new resource. And it will probably happen with the new bundle type, etc.. I don't know how we are going to answer that.
Lloyd McKenzie (Mar 06 2021 at 02:11):
We can 'publish' whatever, whenever. But if we "publish" every 4 months, there'd be little chance of adoption on that cycle. Real-world systems just can't cycle that fast. Plus, anyone who'd implemented a different cycle wouldn't be able to interoperate. I think if we're going to release on a non-regular schedule, those 'special releases' need to be tied to specific, committed implementer communities. That community would then freeze to a release (probably for a period of 18+months) to allow implementation and real-world testing.
The challenge of getting the implementation community to move to a 'new' version is going to be a challenge. The IG should make clear that the solution is temporary. That's about all we can do.
Jean Duteau (Mar 06 2021 at 02:33):
one of the problems is that everyone is still treating FHIR as these monolithic versions. We don't manage the artifacts in the core specification that way but implementers are still implementing the latest official published release. If we published every 4 months, there would obviously be much less change between releases so it might be easier to update your system, especially if the change was in resources that moved from draft to less draft.
Lloyd McKenzie (Mar 06 2021 at 02:57):
No-one is going to churn their system every few months - or want to manage the version-conversion necessary to support all of those versions. We'd end up with a whole lot of implementation silos based on what versions people chose to freeze on.
Gino Canessa (Mar 06 2021 at 03:01):
Lloyd, I feel like we aren't actually that far apart (and this would be easier in a room over a beer =).
Unless I am misreading, your post above is advocating for separating the standard into two parts - a 'stable' and an 'experimental' (for lack of better terms). The 'stable' side should have a long release cycle (I agree), and and the 'experimental' side should have a shorter one (I also agree). Exactly how much shorter is a question I don't think the two of us can solve.
If a resource is stable, it shouldn't matter how many releases happen because it is unchanged*. If a resource isn't stable, it will be changed, but: a) less frequently as it matures (and is implemented), and b) anyone actually implementing that probably wants the changes (e.g., the changes that are in R4B, and many of the changes that aren't).
*assuming we define a way of indicating this; whether it is resource versioning, a logical split in releases, etc.
So even if there were monthly releases (which I am not advocating), most implementers would only care about a fraction of the changes in each one. Yes, generic servers are a different case, and yes, that needs more exploration, but the exact timing on 'experimental' releases feels like an implementation detail at this point in the discussion.
Right now I'd be happy to:
- get wide acceptance that the process needs to be updated
- figure out the shape of the new process
- how are resources separated,
- how do resources move from one category to the next
- how does versioning work
- how can we better support implementers with inter-version compatibility given the new approach
- etc.
(edits: typos)
Jean Duteau (Mar 06 2021 at 04:05):
is everything going to change every 4 months? Wouldn't the FMM4-5 resources not change? And if a resource moved from AE 0-1 wouldn't an implementer want to update that?
(I'll just let Gino handle my side of the discussion from now on)
Lloyd McKenzie (Mar 06 2021 at 04:15):
The "stable" release would contain resources that are either locked into backward compatible mode (normative) or those that aren't yet normative, but are well used and thus the number of breaking changes should be small (and presumably they ought to become normative soon). We can't say that the 'stable' release won't ever have breaking changes though. We can only say that about the normative stuff.
I don't think an implementer would choose to update just because a resource moved from 0 to 1. Implementers in production change when there's a pressing reason to do so or because what they have doesn't work. And given that extensions give you a work-around to most "not work" situations, it'd be pretty uncommon for an implementer who's put a resource from an "unstable" release into production and then make any changes until there was a significant reason to change - i.e. the industry was moving to a new "more stable" release or there were features they wanted that wouldn't have interoperability on the release they'd put out. Keep in mind, that in DSTU2, almost everything was 'unstable', yet a lot of implementers ignored R3 and waited for R4 to move.
Lloyd McKenzie (Mar 06 2021 at 04:17):
The only thing I was pushing back on about Gino's proposal was the notion of tying interim releases to connectathons. I think that's untennable. We might be able to get out 2 releases of unstable content a year, but I think we'd even only do that if we were confident there were specific implementation communities waiting on the changes. Putting out a release when we're not confident there's a community that's going to pick it up benefits no-one.
Peter Jordan (Mar 06 2021 at 04:18):
I like Lloyd's partitioning suggestion. However, we'd have to look beyond US Core in terms of assessing the maturity of resources. If the current targets set by the Global Digital Health Partnership are met, at least 10 countries will have implemented the International Patient Summary IG by the end of this year, and 20 by the end of 2022.
Lloyd McKenzie (Mar 06 2021 at 05:11):
I wasn't focusing so much on the U.S. aspect. The resources referenced in US Core and in IPS are largely the same ones.
Gino Canessa (Mar 06 2021 at 05:50):
Lloyd McKenzie said:
I don't think an implementer would choose to update just because a resource moved from 0 to 1. Implementers in production change when there's a pressing reason to do so or because what they have doesn't work.
I would argue that most implementers don't implement resources at 0. =)
As I mentioned, I'm not too concerned about the timing of the 'fast' releases right now. That will invariably be a longer discussion that results in a compromise of some kind.
John Moehrke (Mar 08 2021 at 13:53):
there should be well publicized points in the calendar when we as an organization decided "which of the unstable is now ready for stable", and that gets integrated at a specific time in the calendar. Drum-beat . I tend to think that this is not more often than once a year, but it should be based on calendar, not ad hoc.
John Moehrke (Mar 08 2021 at 13:56):
would the unstable be broken out into topic areas? or would the unstable be continued to be managed as a blob? Seems this would be best handled as topic spaces. Like we have with Subscription today. This would give the communities interested in those topics something to focus on. The big blob of a release is too hard to focus on.
Lloyd McKenzie (Mar 08 2021 at 15:36):
We could theoretically initiate at a specific time, but given we're tied to balloting process, we don't have a ton of control over when it ends.
Last updated: Apr 12 2022 at 19:14 UTC
