Stream: hapi
Topic: Hapi 510 with forge snapshots
Eeva Turkka (Sep 29 2020 at 12:29):
We are trying to upgrade to Hapi510 and are having trouble using STU3-profiles generated with Forge version 24.2 . The issue seems to be that the snapshots in them are different, the hapi solution generates more elements. The specific mismatching element seems to be at least Extension.extension:name.value[x], both snapshots have elementdefiniton of Extension.extension:name.value[x]:valueString. Is there something we could do to make the snapshots match? Regenerate profiles or configure the hapiserver?
There's a boolean for checkspecials to disable this validation, but it isn't passable from application config.
Jens Villadsen (Sep 29 2020 at 13:03):
HAPI has the tooling for generating snapshots based on differentials. You should either address this to @Ward Weistra or use the snapsnot generation tooling as part of HAPI
Ward Weistra (Sep 29 2020 at 16:30):
@Eeva Turkka Hi Eeva, hard to judge from the current description if there's something wrong with snapshot generation in either of the tool stacks (.NET stack underlying Forge and/or Java stack underlying HAPI). Are the profiles using FHIR 4.0.1? Could you try snapshot generation with the latest version of Forge (25.1.1) and HAPI and compare the two to see if it still occurs?
You're also free to let HAPI regenerate the snapshots, but if there's really a difference in snapshot generation that's good to know so we align the two! Thanks in advance.
Eeva Turkka (Sep 29 2020 at 17:56):
We are using FHIR 3.0.1 for these profiles, since the project is still in STU3 and we will be supporting STU3 for a while still. I will check if there's a difference with the latest version soon. Both the snapshots seem to have the elements Forge generates, Hapi generates those value[x] elements in addition to that -> leading the number of items in the snapshots to mismatch and the profile to be rejected by Hapi validation. The amount of elements by Hapi seems always to be greater than one generated by Forge. (Our profiles are here (https://simplifier.net/finnishphr/~resources?category=Profile) and they seem to cause this error with general Hapi and validation operation on the STU3-branch)
We will try to regenerate the STU3 profiles without snapshots to see if it'll solve this (the snapshot generation tooling wasn't available in Hapi when the project started quite a few years ago so it's not in use yet).
Jens Villadsen (Sep 29 2020 at 18:23):
@Eeva Turkka can the content of the Finnish PHR be downloaded as an IG compatible file - I mean as a tgz file?
Eeva Turkka (Sep 29 2020 at 18:27):
I haven't tried? The profiles, codesystems, valuesets etc are available in Simplifier? I'm just trying to upload a single SD as a post to our FHIR server in this case (and pretty much all of them seem to have this trouble with mismatching snapshots). I will test the differential-based validation tomorrow during office time :).
Eeva Turkka (Sep 30 2020 at 09:41):
We (@Timi Rahikainen ) tested this with new Forge and both 3.0.1 and 3.0.2 and with the hapi.fhir.org server and StructureDefinition validation there and the snapshots generated are mismatched in element count.
The immediate problems with updating profiles can be circumvented with generating the snapshot on the hapi side, we'll need to test a bit and see if there's any impact on perfomance :).
Ward Weistra (Oct 01 2020 at 14:19):
Hi @Eeva Turkka, I've spent quite some time trying to find what you are referring to, but I'm afraid I need more specific examples. For example, can you upload profileX-snapshotFromForge25-1-1.xml and profileX-snapshotFromHAPIx-x.xml and quote the specific differences here?
I've downloaded multiple of your profiles (vital signs and blood pressure), removed the snapshot and regenerated that with Simplifier, Forge and the IG publisher and all of them have a similar set of value[x] elements, as far as I can tell.
Thanks in advance!
Eeva Turkka (Oct 01 2020 at 14:41):
Thank you! We will test some more and get back to this! The trouble is that the hapi generates a new snapshot for the SD for validation in memory temporarily when saving an SD and that doesn't match the ones we have in the profiles in simplifier - they worked fine with 5.0.3 version and with the latest version they stopped being valid without any change to the profiles.
Jens Villadsen (Oct 01 2020 at 14:48):
@Eeva Turkka If I remember correctly the validation framework had quite a bump in functionality from 5.0.x to 5.1.0 in HAPI
Patrick Werner (Oct 01 2020 at 15:58):
@Jens Villadsen totally. Run into several problems/bugs with 5.0.x
5.1.0 just works
Eeva Turkka (Oct 02 2020 at 09:10):
Yup, I noticed and that was part of my original question if there's a workaround for these SDs. There's a "check specials" boolean for the validation that would turn off the snapshot checking for the profiles but it isn't passable from the server configuration because the bean that uses it is created with new inside the validator.
I will try to create a junit test for a profile to reproduce this error.
Morten Ernebjerg (Oct 06 2020 at 09:38):
@Ward Weistra I can reproduce (what looks like) the same error using the latest R4 Forge and the latest validator tool (Java one)
- Load the STU1 IPS Patient profile (v. 1.0.0) into a directory (+ any data type profiles it depends on)
- Use Forge R4 to create a new derived profile from the IPS Patient
- Save the new profile without making any changes (I've attached an example)
- Validate the corresponding StructureDefinition file using the validator
The result is the following error:
FHIR Validation tool Version 5.1.16 (Git# ea0b4c0c1c10). Built 2020-10-01T04:39:09.65Z (5 days old)
Java: 12.0.2 from /Library/Java/JavaVirtualMachines/adoptopenjdk-12.jdk/Contents/Home on x86_64 (64bit). 4096MB available
Paths: Current = /Users/morten.ernebjerg/repos/s4h-fhir-profiles, Package Cache = /Users/morten.ernebjerg/.fhir/packages
Params: -version 4.0.1 profiles/MyPatientUvIps.StructureDefinition.json
Loading
Load FHIR v4.0 from hl7.fhir.r4.core#4.0.1 - 4575 resources (00:03.0623)
Terminology server http://tx.fhir.org - Version 1.0.353 (00:01.0472)
Get set... go (00:00.0003)
Validating
Validate profiles/MyPatientUvIps.StructureDefinition.json Load hl7.fhir.uv.ips#0.3.0 - 81 resources (00:01.0364)
00:00.0551
Done. Times: Loading: 00:05.0317, validation: 00:01.0916
*FAILURE*: 1 errors, 5 warnings, 2 notes
Error @ StructureDefinition : The generated snapshot has a different number of elements 87 that the originally provided snapshot 54
...
Example StructureDefinition: MyPatientUvIps.StructureDefinition.json
Morten Ernebjerg (Oct 06 2020 at 10:17):
PS: Looking at the error trace, it seems the validator is loading the 0.3.0 IPS package (as per discussion in #IPS, the 1.0.0 package is not in the registry at the moment). However, I derived the profile from (a local copy) the 1.0.0 version of the profile. Might that be (part of) the issue?
Ward Weistra (Oct 06 2020 at 11:23):
@Morten Ernebjerg Thanks!
But indeed, if the original snapshot was made based on the IPS patient profile from hl7.fhir.uv.ips#1.0.0 and the validator compares it to a snapshot it generates from the IPS patient profile from hl7.fhir.uv.ips#0.3.0, a difference is most likely due to that, rather than a difference in snapshot generation between Java and .NET tooling.
I wonder how the Java validator chose to use that package and version. Do you specify that in the command calling it (eg. with the -ig flag like in the IG publisher), does it take it from the package.json in the folder from where it's run or does it search the registry for the canonical it found? Would you be able to tell it to use the 1.0.0 version instead if you manually install it to your package cache, eg. with Torinox or by manually placing it there?
If you are able to reproduce the "snapshot has a different number of elements" error with a snapshot generated by Forge and Java validator deriving from the same profile (either both 0.3.0 or 1.0.0) we can investigate any potential snapshot generation differences.
Morten Ernebjerg (Oct 06 2020 at 11:33):
Right, that does indeed seem to be the issue. I got the error without using the -ig parameter. Presumably, what happens is that the validator sees that the profile is derived from the IPS one and then, not having been pointed explicitly to an IG, pulls the most recent corresponding package (v0.3.0). If I add -ig and point it to the folder containing the v1.0.0 IPS profiles, the error disappears.
So this does indeed seem to be a separate issue, sorry for muddying the waters!
Ward Weistra (Oct 06 2020 at 11:52):
@Morten Ernebjerg Not at all! Thank you for helping reproducing errors.
Eeva Turkka (Oct 06 2020 at 14:22):
Ok, so I finally got back to this and wrote a junit test (this is with Hapi 5.1.0):
@Test
public void validate() throws IOException {
String sd = new String(Files.readAllBytes(Paths.get("src/test/resources/fiphr-ext-creatingapplication.json")));
FhirContext dstu3Context = FhirContext.forDstu3();
FhirValidator validator = new FhirValidator(dstu3Context);
ValidationSupportChain validationSupportChain = new ValidationSupportChain(
new DefaultProfileValidationSupport(dstu3Context),
new InMemoryTerminologyServerValidationSupport(dstu3Context),
new CommonCodeSystemsTerminologyService(dstu3Context),
new SnapshotGeneratingValidationSupport(dstu3Context));
FhirInstanceValidator instanceValidator = new FhirInstanceValidator(validationSupportChain);
validator.registerValidatorModule(instanceValidator);
ValidationResult res = validator.validateWithResult(sd);
assertTrue(res.isSuccessful());
}
The json-file is basically this: https://simplifier.net/finnishphr/fiphr-ext-creatingapplication downloaded with the snapshot as json just half an hour ago. The test fails for me with the number of items mismatching, similarily when trying to save the extension tuo a stu3 fhir server.
Grahame Grieve (Oct 06 2020 at 20:00):
I figured out why the IPS 1.0.0 package is not published - a typo in the package-list.json file - and published it. Will get into packages.fhir.org whenever it next updates the registry
Ward Weistra (Oct 07 2020 at 08:18):
Awesome @Grahame Grieve :thumbs_up:
Would you also happen to understand why @Eeva Turkka is seeing a snapshot element mismatch error in the validation unit test she shared above? Her issue is independent of IPS, seeing the error when Java validating resources (eg. this one https://simplifier.net/FinnishPHR/fiphr-ext-creatingapplication) from this project, which only depends on the core STU3 package.
Grahame Grieve (Oct 07 2020 at 23:45):
well, I tried to get the snapshot generation process harmonised between the dotnet code and the java code, but that process is currently waiting on a response from someone on the dotnet side. We have unit tests that we work to, and someone on the dotnet side was going to like at them
Grahame Grieve (Oct 07 2020 at 23:45):
I was waiting here for Eeva to tell me what was actually wrong with the generated snapshot
Eeva Turkka (Oct 08 2020 at 06:51):
Oh, the generated snapshot from Hapi has different amount of elements than from Forge so the profiles that have snashots already with them are now all "invalid" according to Hapi. There seems to be one extra value[x] -element in the hapi generated one (there's a screenshot of the extra element from a debugger in the first message of this discussion), Forge does two elements for the valueString and value and Hapi does three if I looked at it correctly. Trouble is that the two snapshots don't match and the StructureDefinitionValidator rejects the SD. There's the "checkSpecials" boolean that would disable it, but it isn't injectable from outside into the validator.
We've been testing the generated one and there seems to be some issues validating the refrence type of references of a derived profile. Also seems, but we haven't been able to pinpoint it, but there might be some concurrency issues for generating the snapshot for profile under load - cannot confirm the latter one but we are seeing some weird intermittent validation issues (validation fails) for that. It's still under tests so I can't report that one more accurately.
Grahame Grieve (Oct 08 2020 at 09:23):
do you want to post examples?
Eeva Turkka (Oct 08 2020 at 10:37):
Examples of failing profiles? One example is that extension we have in simplifier and I linked - the snapshot it has is invalid according to Hapi. The unit test I posted fails - and it fails pretty much for all the profiles we have in FinnishPHR for same error: mismatching number of elements in snapshot. I used it here as an example because I didn't need to load any other resources for the validation.
Grahame Grieve (Oct 08 2020 at 10:38):
what does 'it is invalid according to HAPI' mean?
Grahame Grieve (Oct 08 2020 at 10:38):
the unit test you added wasn't really a functional test - there's some reasons why the snapshots might be different without affecting the functional behavior
Eeva Turkka (Oct 08 2020 at 10:40):
The snapshots work fine for resource validation, but we are having trouble uploading the profiles since they don't pass the validation of create action?
Eeva Turkka (Oct 08 2020 at 10:41):
The specific error here is "[SingleValidationMessage[locationString=StructureDefinition,message=The generated snapshot has a different number of elements 25 that the originally provided snapshot 15,severity=error]]"
Grahame Grieve (Oct 08 2020 at 11:15):
can you chase that message down? where does it come from?
Eeva Turkka (Oct 08 2020 at 11:17):
It's from line 89 in StructureDefinitionValidator,
if (!snapshot.isEmpty()) {
int was = snapshot.size();
int is = sd.getSnapshot().getElement().size();
rule(errors, IssueType.NOTFOUND, stack.getLiteralPath(), was == is, I18nConstants.SNAPSHOT_EXISTING_PROBLEM, was, is);
}
Grahame Grieve (Oct 08 2020 at 11:55):
oh. ha. That's my fault
Eeva Turkka (Oct 08 2020 at 12:07):
Ok, so it'll be fixed in the code? If needed you are welcome to use our profiles for units :)
Grahame Grieve (Oct 08 2020 at 12:34):
I'm trying to decide what to do about it. but yes, it's something i have to do. The intent of the check is to find out that there's engines out there that aren't aligned, or that mine is not acting reproducibly. But the intent is not to make it other people's problems - it's a discovery thing
Eeva Turkka (Oct 08 2020 at 12:48):
One thing that could help is to make the "checkSpecials" boolean passable trought to the validator from the server so we could just skip those checks?
Grahame Grieve (Oct 08 2020 at 13:00):
yes something like that. but I have to review exactly how.
Vikrant Chauhan (Oct 08 2020 at 13:02):
Grahame Grieve said:
yes something like that. but I have to review exactly how.
Much appreciated! @Grahame Grieve
Last updated: Apr 12 2022 at 19:14 UTC
