FHIR Chat · Approach 4A Versioning Policy · tooling

Stream: tooling

Topic: Approach 4A Versioning Policy

---

Grahame Grieve (Jun 03 2020 at 21:29):

Place holder for discussion to come

Grahame Grieve (Jun 03 2020 at 22:59):

see https://chat.fhir.org/#narrow/stream/179240-Announcements/topic/Release.20R4A

Grahame Grieve (Jun 03 2020 at 23:04):

The original proposal for an interim R4 release was for 4.1 - that would be 4.1.0. 4.1.0 has already existed as the version of the current build after 4.0.0 was published, but there is no remaining evidence of that except in my memory (and maybe a few others).

So we could release an interim release as 4.1.0. Tools and reference implementations would need to know what 4.1.0 is, and how it does and doesn't differ from 4.0.0. My own intention would be to drop support for 4.0 internally and just treat it was 4.1.0 (though still supporting it as a stated version)

But we have no way to get to a 4.1.0 release under the current versioning rules, since there would be at least 4 staging releases to get there. The current versioning rules don't give us space to do it. So we'd have to change them somehow, and at least the following tools/libraries would need to handle this somehow:

• NPM package specification
• FHIR core / Validator / IG publisher
• tx.fhir.org + packages.fhir.org
• reference implementations (dotnet etc)
• Forge? Simplifier?

Grahame Grieve (Jun 03 2020 at 23:04):

I'm not sure how we would go about this.

Grahame Grieve (Jun 03 2020 at 23:06):

@Ewout Kramer @Ward Weistra @James Agnew @Josh Mandel @Martijn Harthoorn @Chris Moesel @Lloyd McKenzie @Mark Iantorno at a minimum should all comment on this please

Josh Mandel (Jun 03 2020 at 23:10):

Just making sure I understand that initial question. Are you saying that we would need staging releases and each one would sort of take up a version slot at the minor position? so the lowest number version we could get to through our existing process would be 4.5.0?

Grahame Grieve (Jun 03 2020 at 23:13):

we're using 4.X for the r5 development - so we've already published 4.2 and 4.4

Grahame Grieve (Jun 03 2020 at 23:14):

we could intersperse r5 and r4a, so that 4.6 is the first draft of what would become r4a

Grahame Grieve (Jun 03 2020 at 23:14):

that sounds like the worst of all worlds to me

Josh Mandel (Jun 03 2020 at 23:14):

Yeah.

Josh Mandel (Jun 03 2020 at 23:14):

I'm just trying to understand whether/why all of our staging releases require minor version numbers.

Grahame Grieve (Jun 03 2020 at 23:15):

because they represent changes to structures and so are not wire format compatible, and that's how the tools decide what versions they support

Grahame Grieve (Jun 03 2020 at 23:16):

we could change that. If we wanted. It would mean that the tools couldn't work that way. But whatever we do breaks the tools somehow

Grahame Grieve (Jun 03 2020 at 23:17):

we could say that the first draft of 4a would be 4.0.2, for instance. But the tools would look at that and say 'under the current rules, that's means it's a patch on 4.0.1 and wire format compatible'. And, btw, we already patched 4.0.0 to 4.0.1

Grahame Grieve (Jun 03 2020 at 23:17):

and the tools know that they treat 4.0.0 as 4.0.1 for all purposes except for a few since it's a patch change

Grahame Grieve (Jun 03 2020 at 23:18):

and we do have requests for 4.0.2 already too. So we'd be interspersing something different

Josh Mandel (Jun 03 2020 at 23:19):

Yeah this doesn't feel right either.

Chris Moesel (Jun 03 2020 at 23:19):

If we're going to break things for sure, now would be a good time to seriously consider semantic versioning... In most people's semantic versioning implementations, pre-releases are done via labels. So, for example, if we were marching toward 5.0.0, we would have started at 5.0.0-alpha.1, then 5.0.0-alpha.2 (or 5.0.0-beta.1), and ballot versions would be 5.0.0-rc.1, etc. I'm not sure that helps us right now w/ the 4.x stuff though.

Chris Moesel (Jun 03 2020 at 23:21):

But if we followed that, then the rules for official releases are still very tight and predictable. And rules for pre-releases would basically be "expect that anything can change".

Josh Mandel (Jun 03 2020 at 23:22):

That'd let us squeeze in 4.1.0-alpha1 etc as as needed.

Mark Iantorno (Jun 03 2020 at 23:31):

I don't think I understand completely... How long lived would this version be? Would it be the only r4 version? Or would we now be maintaining both an r4 and the r4-alpha as well?

Grahame Grieve (Jun 03 2020 at 23:35):

well, we do use semantic versioning as we can but we did not adopt the pre-release system since that's never quite what we were doing

Grahame Grieve (Jun 03 2020 at 23:35):

but it still doesn't solve the problem we have now, either

Grahame Grieve (Jun 03 2020 at 23:36):

because every version would be 4.1.0-something, or 5.0.0-something and there'd never be anything else other than them and 4.1.0 and 5.0.0

Chris Moesel (Jun 03 2020 at 23:43):

Well, in theory you still might have 4.1.1 and 5.0.1 too (official patch releases). But semantic versioning is pretty clear that minor increments represent backwards-compatible changes (i.e., no breaking changes) and thus far FHIR has not followed that. As Josh noted, if we did adopt it, it gives us a way to get to 4.1.0 while also having staging releases (using the labels). But given the current state of things, it would still be messy for a while.

Grahame Grieve (Jun 04 2020 at 00:03):

as I said, we use it as we can. What we actually have - and document - is publication.major.minor.patch. So it's semver, but off by one, and with a publication cycle added.

We used to have patch, which was the svn version number (e.g. v1.0.0-6850 for http://hl7.org/fhir/2015Sep/), but we lost that when we moved to github, and so we dropped it quietly, though some of you might remember how discomforted I was about that

Grahame Grieve (Jun 04 2020 at 00:03):

so 4.1 is actually first major release after R4.

Ward Weistra (Jun 04 2020 at 07:17):

I agree semver would be a great solution, as I understand it. So, for going from R5 to R6:

• R5 gets released: 5.0.0 (after coming from 5.0.0-prerelease6 or 5.0.0-rc7)
• Development for a patch release releases an evaluation version: 5.0.1-prerelease1 or 5.0.1-rc1
• R5 patches (backwards compatible bugfixes) are released: 5.0.1
• Development for new feature release releases an evaluation version: 5.1.0-prerelease1 or 5.1.0-rc1
• R5 gets more (backwards compatible) functionality (like the current proposal, assuming this is all backwards compatible): 5.1.0
• Development for R6 releases an evaluation version: 6.0.0-prerelease1 or 6.0.0-rc1 or 6.0.0-wgm042022

Ward Weistra (Jun 04 2020 at 07:23):

Anything with -whatever behind it will be treated as a prerelease version. So, it's easy for all tooling to warn users when they are depending on those (especially if non-prerelease newer versions are available).

Ward Weistra (Jun 04 2020 at 07:25):

Similarly, we can warn any user which doesn't use the latest patch or minor version and inform when there is a newer major version of the FHIR core package (and any other package) they are depending on.

Ward Weistra (Jun 04 2020 at 07:31):

For R4 we currently seem to have the following official version numbers:

• 4.0.0 - Archived
• 4.0.1 - FHIR Release #4
• 4.2.0 - FHIR Release #5: Preview #1
• 4.4.0 - FHIR Release #5: Preview #2

In published packages there's

• 4.0.1 for hl7.fhir.r4.core.
• 4.2.0 and 4.4.0 for hl7.fhir.r5.core. Both not marked as a prerelease in the semver way. (And not in the registry, since the FHIR versions aren't supported yet)

Ward Weistra (Jun 04 2020 at 07:44):

Since there's no published packages for hl7.fhir.r4.core beyond 4.0.1 we could just release 4.1.0-rc1 or 4.1.0 next. Going forward we could avoid reusing 4.2.0 and 4.4.0 as a version number for R4 (just skip over those minor versions if we get there).

And unlist the current two packages for hl7.fhir.r5.core and republish them as:

• 4.2.0 -> 5.0.0-prerelease1
• 4.4.0 -> 5.0.0-prerelease2

John Moehrke (Jun 04 2020 at 13:27):

so what is the problem with 4A being 4.5 or 4.6? Are we all that wrapped up in worrying that people are going to see this and bother us wondering where 4.1, 4.2, 4.3, and 4.4 went? Surely we can take that kind of questions.

John Moehrke (Jun 04 2020 at 13:28):

or what about 4.10.0 really mess with their minds

Chris Moesel (Jun 04 2020 at 13:45):

We could go all Apple on them and start using roman numerals.

John Moehrke (Jun 04 2020 at 13:45):

or microsoft and use the year -- 2020 will already be remembered

Ward Weistra (Jun 04 2020 at 14:48):

Indeed, we could divorce the 'marketing version number' from the actual under the hood version number.
I do think 4.5.0 not being based on 4.4.0 and being closer in similarity to 4.0.1 to be very confusing.

David Hay (Jun 06 2020 at 03:06):

or microsoft and use the year -- 2020 will already be remembered

Not entirely sure that we want FHIR associated with 2020 myself...

Grahame Grieve (Jun 07 2020 at 00:29):

why not? It's a great year. Going awesomely well

Lloyd McKenzie (Jun 07 2020 at 14:56):

Memorable, at least...

Ward Weistra (Jun 09 2020 at 06:30):

I think package hl7.fhir.core 4.0.1(http://hl7.org/fhir/R4/package.tgz) illustrates why it would be preferable to have the patch version part for packages still available for package-only fixes: It has dependencies on hl7.fhir.r3.core etc. which clearly were meant to be hl7.fhir.r4.core. But now, if we want to keep package version and FHIR version in sync, we can't update that package without a new FHIR release. FHIR#27792

Grahame Grieve (Jun 09 2020 at 06:56):

sigh. Typo I made yesterday :-(

Grahame Grieve (Jun 09 2020 at 06:56):

they should most definitely be r4 not t3

Grahame Grieve (Jun 09 2020 at 06:58):

actually, it wasn't yesterday

Ewout Kramer (Jun 09 2020 at 10:02):

Yes, it seems the decision we made years ago that the R5 "WGM previews" are still numbered 4.x are a cause of this versioning problem . Grahame - we did thoroughly discuss this back then - what were our reasons to chose 4.x for early R5 releases, do you remember?

Ewout Kramer (Jun 09 2020 at 10:05):

Note that none of this of course fixes the problem around publishing a new version - we get pushback that we automatically upgrade 4.0.0. to 4.0.1 in Forge, so whatever guise we choose to publish R4A under, it will feel like publishing a new (major) version I am afraid.

Ewout Kramer (Jun 09 2020 at 10:07):

I wonder if some of it can be solved by allowing IGs to publish new resources. I know this will hurt, since we have this fixed ResourceType and FhirType enumerations everywhere, but I could then publish at least those new medication resources as an addon to 4.0.1, instead of a new version.

Ewout Kramer (Jun 09 2020 at 10:08):

(also, problably you need to add some of the newer types to existing Resource Reference typerefs - well, there might be more of these "little" issues)

Grahame Grieve (Jun 09 2020 at 11:11):

it won't be little issues

Grahame Grieve (Jun 09 2020 at 22:03):

what were our reasons to chose 4.x for early R5 releases

So that r5 would be 5.0.0

Grahame Grieve (Jun 09 2020 at 22:04):

the fundamental problem is that semver doesn't have a versioning strategy for forking, which was what 4A is

Grahame Grieve (Jun 09 2020 at 22:05):

e.g. in classic semver, someone asks you to make a breaking change to 3.1.3, after you've gone and already published 3.1.4, 3.1.5, and all of 3.2.0 -> 3.2.8 (and 3.4.x and 4.0.x as well)

Grahame Grieve (Jun 09 2020 at 22:05):

what will the version be?

Jean Duteau (Jun 09 2020 at 22:14):

doesn't that depend on whether you're introducing the same change across all of your versions? i.e. 3.1.6, 3.2.9, 3.4.(x+1), 4.0.(x+1)?

Grahame Grieve (Jun 09 2020 at 22:24):

what if you're not

Grahame Grieve (Jun 09 2020 at 22:36):

@Ward Weistra I can fix the package at http://hl7.org/fhir/R4/package.tgz but that won't matter, right? is there a way forward to fix this without a whole new release of R4?

Chris Moesel (Jun 10 2020 at 01:48):

e.g. in classic semver, someone asks you to make a breaking change to 3.1.3, after you've gone and already published 3.1.4, 3.1.5, and all of 3.2.0 -> 3.2.8 (and 3.4.x and 4.0.x as well)

In theory (or perhaps in an ideal world), that's not a request that someone makes. If 3.1.x are truly patch releases, and 3.x are truly backwards-compatible minor releases, there is no reason for someone to insist on being on 3.1.3. If semver is working, anyone on 3.1.3 should be able to be on the latest 3.x (3.4?) without any trouble. So a breaking change would go to the next major number. BUT... that is an ideal world, and that does fit better for software than for specs, so... point taken.

Michael Lawley (Jun 10 2020 at 02:13):

4.a.0 ?

Vassil Peytchev (Jun 10 2020 at 04:18):

I think the mixing of build versions (4.2.0, 4.4.0) and release versions (4.0.0, 4.0.1) is at the root of the problem.

What if, in addition to the CI Build master branch, we had a publish branch, and each PSS manages its own branch,
periodically merging their changes into master, and when ready to ballot, merge into the publish branch, then create the ballot branch from there for ballot reconciliation, and eventually publish the new release.

I understand that this probably is not taking into account any tooling issues, but is it too hard of a problem to solve?

Just to figure out some of these things for myself, I am trying to build the R4Final branch locally, and then will attempt to apply the 4A candidates to that branch to see if the result is worth the effort...

Grahame Grieve (Jun 10 2020 at 06:57):

well, you've mixed up different issues there - one is the process for how to achieve something, and the other is the way to label what is achieved

Michael Lawley (Jun 10 2020 at 06:58):

4.a.0 is a serious proposal.
As things stand, the first two "digits" of the x.y.z really indicate the Major version of FHIR (since changes to either x or y indicate a breaking change).
A consequence of this is that order is unimportant (for machines). I would further posit that nobody necessarily depends on the semver elements being numbers.
Introducing 'a' not only escapes the strangeness of 4A being closer to 4.0.1 than 4.2.0 or 4.4.0, but it also resonates with "4A" and clearly distinguishes it as different

Grahame Grieve (Jun 10 2020 at 07:01):

I don't believe it's totally true that order is not important. It's truer that we could define it as not important, maybe

Grahame Grieve (Jun 10 2020 at 07:02):

But I think it creates my some tooling difficulties, in principle, because I can code to specific decisions and use order dependent logic for whether they apply

Ward Weistra (Jun 10 2020 at 10:16):

@Grahame Grieve I think the only real solution would be releasing a 4.0.2 package and separating package (patch) version from FHIR release... Using the R4.0 FHIR standard? Then take the latest 4.0.x version of hl7.fhir.r4.core.

I think semver supports branching, as long as you only use the stable version numbers for stable releases and every backwards incompatible change ups the major version (Or maybe we could stretch that to 'up the major or minor' version for FHIR).

• So, any technical correction version of FHIR 4.0 comes out as 4.0.x. And since those changes are backwards compatible users can always use the latest.
• Anyone introducing backwards incompatible changes compared to the latest stable release should up the major (or minor) version number, eg 4.1.x. And if releases are made that are not yet stable (like for WGMs) they should get labeled as such: 4.1.x-preview1 or 4.1.x-mybranch. The latter part should only be removed when its ready for public release.

Ward Weistra (Jun 10 2020 at 10:19):

@Michael Lawley I wonder whether standard NPM servers or semver libraries would be able to handle characters in the version numbers (except in the postfix part).

Grahame Grieve (Jun 10 2020 at 12:13):

I think semver supports branching

As long as you only branch in some ways

Grahame Grieve (Jun 10 2020 at 12:13):

and we are talking about making the latest changes in a way that doesn't meet those conditions

Grahame Grieve (Jun 10 2020 at 12:14):

but I think that the proposal isn't going to fly; there's too many problems. I think we'll just STU the first publication of R5 and publish a reconciled version as the STU for 4B and also the first normative ballot for R5

Grahame Grieve (Jun 10 2020 at 12:15):

this doesn't mean that we can't revise the way we do versioning, but it does mean that the branching thing is not the driver

Joel Schneider (Jul 22 2020 at 20:27):

At some point, it might be interesting to consider adopting something like the Linux kernel version numbering convention, where an even-numbered minor version indicates a stable release, and an odd-numbered minor version indicates a development release.

Grahame Grieve (Jul 22 2020 at 20:43):

well, we're pretty much following that now.

Grahame Grieve (Jul 22 2020 at 20:43):

though it's not formal

---

Last updated: Apr 12 2022 at 19:14 UTC