Stream: IG creation
Topic: Comparison with previous versions
Grahame Grieve (Jul 24 2020 at 21:50):
The next version of the IG Publisher will add a new feature: automatic comparison with previous versions of the specification. By default, the IG publisher will automatically compare with the last published version.
Grahame Grieve (Jul 24 2020 at 21:50):
the first thing to note is that the QA checks will change. Here's the QA check for davinci-deqm:
Grahame Grieve (Jul 24 2020 at 21:50):
Grahame Grieve (Jul 24 2020 at 21:51):
ignore the version check on the validator- that's running my local dev version, so it's all wrong.
Grahame Grieve (Jul 24 2020 at 21:51):
note the dependency analysis. This will matter in the future for FMG. So make sure your ducks are in a row on that one
Grahame Grieve (Jul 24 2020 at 21:52):
A typical comparison will look like this:
Grahame Grieve (Jul 24 2020 at 22:04):
Grahame Grieve (Jul 24 2020 at 22:04):
And a typical profile comparison:
Lloyd McKenzie (Jul 24 2020 at 22:09):
Can/should the template have a tab for profiles and other artifacts that shows the comparison?
Grahame Grieve (Jul 24 2020 at 22:11):
certainly not yet while we shake the comparison down. I've put a lot of work into it at ONC request but this is start of alpha testing. Maybe once it's stable and proven, and we know how it impacts implementers, we can talk about what that would look like. One complication is what I'm about to explain:
Grahame Grieve (Jul 24 2020 at 22:12):
Grahame Grieve (Jul 24 2020 at 22:12):
and
Grahame Grieve (Jul 24 2020 at 22:12):
Grahame Grieve (Jul 24 2020 at 22:16):
how to manage this:
- if you do nothing, the IG publisher will automatically do a comparison with the last pulished version (what ever it's status)
- you can configure this by putting a parameter "
version-comparison" in your ImplementationGuide resource. - if this has the value
{last}, that means that last published version (that's the default) - if this has the value
{current}, that means that last published version that is marked as the current version in {canonical}/package-list.json - if this has the value
n/a, the no version comparison will be performed - otherwise, the value can be any published version e.g.
1.0.0
Grahame Grieve (Jul 24 2020 at 22:17):
this parameter can be present more than once, so that you get more than one comparison
Grahame Grieve (Jul 24 2020 at 22:17):
note to editors: like all other parts of the infrastructure, this uses the value set caching infrastructure. The first time a comparison is run, it may take some time, but after that, the comparison is fairly quick
Grahame Grieve (Jul 24 2020 at 22:26):
@Lloyd McKenzie what might be useful for implementers is the dependency analysis. but for now, that's an artifact of the qa system, and there's no way to get it into a template
Oliver Egger (Aug 03 2020 at 08:19):
Grahame Grieve said:
- if this has the value
{current}, that means that last published version that is marked as the current version in {canonical}/package-list.json
we have now to different "current" meanings depending on the context, is this correct?
- a dependency/reference to an ig with #current is refering to latest version of the IG as built by the IG auto-publisher (see above) - usually this is updated when a commit is pushed to github (master???)
- current for comparision as mentioned above which is the published ig marked as current
Grahame Grieve (Aug 03 2020 at 12:15):
yes that's right 2 - 2 different meanings, but in very different contexts. Actually, we already had this meaning for current in package.json
Bryn Rhodes (Aug 05 2020 at 07:11):
I think that I'm getting a null reference exception on this feature, because the previously published version of the IG I'm working on is based on an interim snapshot (v3.5.0) of FHIR and the version resolution code doesn't seem to like that version. The call on this line: https://github.com/HL7/fhir-ig-publisher/blob/master/org.hl7.fhir.publisher.core/src/main/java/org/hl7/fhir/igtools/publisher/PreviousVersionComparator.java#L210 is returning a null id for the VersionUtilities.packageForVersion call.
Bryn Rhodes (Aug 05 2020 at 07:11):
So I tried disabling the version checking with the version-comparison parameter, but no joy, it still fails on that line.
Bryn Rhodes (Aug 05 2020 at 07:15):
This is the PDDI-CDS IG: https://github.com/HL7/PDDI-CDS
Bryn Rhodes (Aug 05 2020 at 07:17):
And I just committed the txcache so it shouldn't take an hour to get to that error.
Grahame Grieve (Aug 05 2020 at 20:41):
ok fixed. Takes 5min for me - still not enough cached, but I'm not sure why and don't have time to investigate
Grahame Grieve (Aug 05 2020 at 21:08):
@Bryn Rhodes I do see some invalid UCUM codes...
Bryn Rhodes (Aug 05 2020 at 21:09):
Yes, team is fixing those along with the other errors. Thanks Grahame!
Grahame Grieve (Aug 05 2020 at 21:16):
also, the template doesn't meet requirements
Grahame Grieve (Aug 05 2020 at 21:20):
also, this: downloads.html error The html source references validator.pack which is deprecated. Change the IG to describe the use of the package system instead
Rich Boyce (Aug 06 2020 at 14:56):
Thank you Grahame, Bryn, and all for your help!
Mark Kramer (Oct 31 2020 at 14:14):
I posted something on the Implementers stream that should have been here. As I said there, the comparison feature is amazingly useful. Kudos to @Grahame Grieve. But at the same time, I'm noticing some limitations in the current version. It doesn't seem to pick up cardinality changes, and it counts some things as breaking changes that really aren't (such as upping the version number). I will make Git issues for them.
Grahame Grieve (Nov 02 2020 at 20:36):
ok thanks. I'll look at them
Brian Reinhold (Jan 26 2021 at 10:12):
Grahame Grieve said:
also, this: downloads.html error The html source references validator.pack which is deprecated. Change the IG to describe the use of the package system instead
I am getting that error. How do I fix it?
Lloyd McKenzie (Jan 26 2021 at 14:36):
Update the text of your IG to no longer refer to validator.pack. Point to the .tgz file instead.
Eric Haas (Jan 18 2022 at 00:37):
@Grahame Grieve is it possible to use this to compare the latest US Core with Argonaut STU2?
Grahame Grieve (Jan 18 2022 at 00:50):
not the automatic one, no. you'll have to do it manually for that
Eric Haas (Jan 18 2022 at 00:54):
Is there a tool to do that ?
Eric Haas (Jan 18 2022 at 00:56):
I looked at the validator to compare profiles, but that seems to produce a different output
Eric Haas (Jan 18 2022 at 01:25):
Does anyone think a CSV (or Excel) version of these would be useful for download and offline processing?
Grahame Grieve (Jan 18 2022 at 01:35):
it's the same code
Eric Haas (Jan 18 2022 at 02:02):
Grahame Grieve said:
it's the same code
... to compare US Core with Argonaut? Great! it is documented somewheres?
Max Masnick (Jan 18 2022 at 21:14):
@Eric Haas I presented to FHIR-I today on a custom tool we made to produce the Excel data dictionary for mCODE. One of the nice side-effects of this was a way to do an element-level diff of two arbitrary IG versions. If the differential at the bottom of http://build.fhir.org/ig/HL7/fhir-mCODE-ig/dictionary.html looks helpful, let me know — we will be open sourcing the code that does this soon
Eric Haas (Jan 18 2022 at 23:24):
Thanks, @Max Masnick - I would like to use this, Is it able to compare US Core to Argo DSTU2 i.e is across IGs ( different base urls ) and will it be available before June?
Max Masnick (Jan 19 2022 at 00:47):
It is able to compare across different IGs, but you'd need to provide a manual mapping of profiles between the IGs -- it provides the element-level comparison profile by profile.
It should be open sourced in the coming weeks...not sure exactly how long but certainly before June.
Eric Haas (Jan 19 2022 at 03:53):
How much coding will I need to do ( hoping is a npm or something)
Declan Kieran (Mar 03 2022 at 22:34):
Max Masnick said:
It is able to compare across different IGs, but you'd need to provide a manual mapping of profiles between the IGs -- it provides the element-level comparison profile by profile.
It should be open sourced in the coming weeks...not sure exactly how long but certainly before June.
Hi, did you open source the code for this. It looks like it would be very useful for some work I am doing comparing various profiles. Thanks!
Grahame Grieve (Mar 03 2022 at 23:50):
have you tried using the validator for this?
Declan Kieran (Mar 04 2022 at 08:40):
Grahame Grieve said:
have you tried using the validator for this?
@Grahame Grieve
I just came across that ability after I posted this. I did get it working for profiles of the same version and it works really well. It mentions that it can compare across versions, but when I tried profiles with different versions, 3.0 and 4.0 in this case, depending on which version is set, it doesn't find the other one, saying it can't find right or left. Should it support this? Thanks.
Declan Kieran (Mar 04 2022 at 09:10):
I should also add that I tried removing the version parameter when trying different version profiles, but it didn't seem to like that
Lloyd McKenzie (Mar 04 2022 at 14:38):
In theory, if your profile references are version-specific it should work, but I don't know that it's been well-exercised.
Grahame Grieve (Mar 07 2022 at 23:50):
it should work. what is your parameters? @Declan Kieran
Declan Kieran (Mar 08 2022 at 11:09):
Grahame Grieve said:
it should work. what is your parameters? Declan Kieran
It does seem like an issue with the parameters.
It will generate a comparison when the two profiles using this invocation
java -jar validator_cli.jar -compare -dest compare_folder -version 3.0 -ig CareConnect-AllergyIntolerance-1.json -ig CareConnect-GPC-AllergyIntolerance-1.json -left https://fhir.hl7.org.uk/STU3/StructureDefinition/CareConnect-AllergyIntolerance-1 -right https://fhir.nhs.uk/STU3/StructureDefinition/CareConnect-GPC-AllergyIntolerance-1
I then tried it with an STU3 profile and an R4 profile, just removing the version parameter like this
java -jar validator_cli.jar -compare -dest compare_folder_r4 -ig CareConnect-AllergyIntolerance-1.json -ig UKCoreAllergyIntolerance-withsnapshot.json -left https://fhir.hl7.org.uk/STU3/StructureDefinition/CareConnect-AllergyIntolerance-1 -right https://fhir.hl7.org.uk/StructureDefinition/UKCore-AllergyIntolerance
and it does seem like it doesn't like to order of the parameters, I've also tried rearranging them to no avail.
...
Params: -compare -dest cc-ukc-alg-com -ig CareConnect-AllergyIntolerance-1.json -ig UKCoreAllergyIntolerance-withsnapshot.json -left https://fhir.hl7.org.uk/STU3/StructureDefinition/CareConnect-AllergyIntolerance-1 -right https://fhir.hl7.org.uk/StructureDefinition/UKCore-AllergyIntolerance
Scanning for versions (no -version parameter):
..Detect format for CareConnect-AllergyIntolerance-1.json
..Detect format for UKCoreAllergyIntolerance-withsnapshot.json
********************
* The file name you passed in, '-compare', doesn't exist on the local filesystem.
* Please verify that this is valid file location.
********************
Exception in thread "main" java.io.IOException: File -compare does not exist
at org.hl7.fhir.validation.ValidatorUtils.extractReferences(ValidatorUtils.java:131)
at org.hl7.fhir.validation.ValidatorUtils.parseSources(ValidatorUtils.java:155)
...
Good to know it should support it though!
Declan Kieran (Mar 08 2022 at 14:42):
@Grahame Grieve I've got past the error using the debugger
In IgLoader.scanForVersions, the sources that is passed contains all the switches, but I think it only wants the StructureDefinitions provided in the -ig switches as it then bombs on ValidatorUtils.parseSources. Modifying this on the fly gets me past that error, I now have issues of content not being fetched because URL's not resolving in the profiles, but that's something I think I'll have to figure out myself! Cheers : )
edit:
Ah, spoke to soon, sorted the URL's, but now getting
...
fhirVersion in https://fhir.hl7.org.uk/STU3/StructureDefinition/CareConnect-AllergyIntolerance-1: 3.0
fhirVersion in https://fhir.simplifier.net/HL7FHIRUKCoreR4/StructureDefinition/Extension-UKCore-AllergyIntoleranceEnd: 4.0
Exception in thread "main" java.lang.Exception: -> Multiple versions found. Specify a particular version using the -version parameter
at org.hl7.fhir.validation.cli.services.ValidationService.determineVersion(ValidationService.java:336)
at org.hl7.fhir.validation.cli.services.ValidationService.determineVersion(ValidationService.java:314)
at org.hl7.fhir.validation.ValidatorCli.doLeftRightComparison(ValidatorCli.java:221)
at org.hl7.fhir.validation.ValidatorCli.main(ValidatorCli.java:160)
Grahame Grieve (Mar 15 2022 at 06:27):
hmm where can I find the source for that so I can reproduce?
Declan Kieran (Mar 23 2022 at 12:18):
Grahame Grieve said:
hmm where can I find the source for that so I can reproduce?
I was used the source of the validation project from the org.hl7.fhir.core repo. I just tried again there and it seems to be the same. I've attached a zip file with a bash script that downloads the latest prebuilt validator_cli and runs the different commands on the files I was talking about, along with some inline comments on what exceptions happened for me and what I did to debug through the code to get round the parameter error. Thanks! example.tar
Grahame Grieve (Mar 30 2022 at 01:52):
well, that's a r3 profile, but you specified to use R4
Declan Kieran (Mar 31 2022 at 08:34):
Grahame Grieve said:
well, that's a r3 profile, but you specified to use R4
The CareConnect-AllergyIntolerance-1 is R3 and the UKCore-AllergyIntolerance is R4, in the cmd.sh I've tried setting the version switch to 3.0 and 4.0 and removing it when calling the validator, with different exceptions. The exceptions are in the comments in the shell script. I also tried adding in a generated snapshot to the UKCore-AllergyIntolerance profile to see if that would make a difference, and these are the commands at the bottom of that shell script, although not sure why I thought that would help! Regardless all the same exceptions were raised.
Grahame Grieve (Mar 31 2022 at 22:58):
well, that's how it's documented to behave. But it's not really practical. I'll have to think about how to manage that. All the code exists, but it's not obvious how to do the command line thing
Grahame Grieve (Mar 31 2022 at 22:58):
@David Otasek we need to talk about this one
David Otasek (Apr 06 2022 at 19:05):
After some discussion and double-checking some pitfalls, I think the following syntax may be appropriate:
-ig [3.0]{path}
[ would not be allowed as the starting character of the scheme portion of a URL, which means it wouldn't invalidate the use of any existing URLs. Technically, a filename COULD start with [ in multiple OSs, however, it doesn't seem like a likely situation, and the workaround would be pretty simple for the user.
Initially, I thought an 'option for an option' approach could work (something like -ig-fhir=3.0 {path}, but looking at the standards in GNU, for example, that is generally outside of the norm for both the name of the option, and generally, whatever comes after the option is automatically its argument. So in our case '-fhir=30' would be interpreted as the argument by convention, and {path} would not necessarily be associated with anything.
Grahame Grieve (Apr 06 2022 at 22:16):
-ig [3.0]{path} works for me
Last updated: Apr 12 2022 at 19:14 UTC
