Stream: hapi
Topic: Error loading de.gecco package into JPA starter
Morten Ernebjerg (May 12 2021 at 14:10):
I am trying to load the de.gecco#1.0.3 FHIR package into the JPA starter server by declaring it in application.yaml. The loading is triggered just fine, but at some point an error is thrown, seemingly because of a problem with a dependency (of which this package has quite a few). The triggering error is this (I have included more complete log output in the expandable spoiler section below):
Caused by: ca.uhn.fhir.rest.server.exceptions.PreconditionFailedException: Unknown base definition: https://www.medizininformatik-initiative.de/fhir/core/modul-diagnose/StructureDefinition/Diagnose
The profile with this URL is indeed included in the de.medizininformatikinitiative.kerndatensatz.diagnose#1.0.1 package which is declared as a dependency. That particular version of this package is annotated as "unlisted" in the package registry, but is still downloadable from simplifier. Might that be the source of the problem in HAPI or is there perhaps an issue with the dependency resolution?
Relevant part of server log
Grahame Grieve (May 13 2021 at 00:23):
I think you'll have to step it to find out exactly where it goes wrong
Morten Ernebjerg (May 13 2021 at 19:09):
OK, will spend some serious time with the debugger when I get back to the office Monday and see if I can find the culprit :magnifying_glass: .
Morten Ernebjerg (May 17 2021 at 11:32):
So, going down the rabbit-hole I found the following: The overall IG load failure error flies in the method PackageInstallerSvcImpl#install when it finds a profile (directly in the package) without a snapshot, tries to generate this snapshot, and than cannot find a profile that the profile in question is based on (re-profiling of). Specifically, the underlying error happens here in SnapshotGeneratingValidationSupport#generateSnapshot because ValidationSupportChain#fetchStructureDefinition does not find the StructureDef in any of the validation supports. In my case, the specific chain of re-profiled profiles that triggered this was the following:
https://www.netzwerk-universitaetsmedizin.de/fhir/StructureDefinition/cardiovascular-diseases- is directly in package, is a re-profiling of...https://www.netzwerk-universitaetsmedizin.de/fhir/StructureDefinition/gecco-base-condition- is directly in package, is a re-profiling of...https://www.medizininformatik-initiative.de/fhir/core/modul-diagnose/StructureDefinition/Diagnose- only present in a dependency (de.medizininformatikinitiative.kerndatensatz.diagnose#1.0.1)
The last profile is not found, triggering the error. So it looks like the code tries to use resources from the dependencies without having pulled them first (I couldn't see where in PackageInstallerSvcImpl#install the dependencies should actually be pulled).
Morten Ernebjerg (May 17 2021 at 11:34):
(PS: As noted in the initial message, the package de.medizininformatikinitiative.kerndatensatz.diagnose#1.0.1 is unlisted in the package registry)
Jens Villadsen (May 18 2021 at 20:57):
would you have some example packages 'to spare' in order to debug it?
Morten Ernebjerg (May 19 2021 at 05:00):
@Jens Villadsen Thanks for looking into this - the one case I have seen is triggered by loading the de.gecco#1.0.3 package . Is that enough? I don't have any "synthetic" packages triggering the behaviour at hand, though I suspect any package with a snapshot-less profile that is based on a profile in a dependency would do the trick.
Morten Ernebjerg (May 25 2021 at 08:57):
Ahoi @Jens Villadsen :wave: , did you by any chance find time to look at this or is there perhaps also still input from me missing?
Jens Villadsen (May 25 2021 at 09:07):
I haven't had the time to look at it
Jens Villadsen (May 25 2021 at 09:28):
Now I have ... you might wan't to bring @James Agnew in on this
Morten Ernebjerg (May 25 2021 at 09:37):
Thanks for ringing his doorbell :smile:
Jens Villadsen (May 25 2021 at 18:04):
If find the time, ill have a look as well
Morten Ernebjerg (May 26 2021 at 09:34):
Update: Problem persists (missing resources from package dependencies at snapshot generation), even with the new updated package de.gecco#1.0.4-rc.1 which I believe does not use unlisted packages.
Morten Ernebjerg (May 26 2021 at 10:50):
(It seems one can get around the issue by manually declaring the dependencies for loading, but the JDoc for the ca.uhn.fhir.jpa.packages#install method explicitly says Installs a package and its dependencies., suggesting that dependency handling is included. Which is, I realize, no picnic cf. this other thread on dependency conflicts I opened)
Jens Villadsen (May 26 2021 at 10:53):
I don't follow ... the loading of packages are done recursively. See https://github.com/hapifhir/hapi-fhir/blob/75c74a22dbd1f0dde3631b540d1898eef2a2666f/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/packages/PackageInstallerSvcImpl.java#L242
Morten Ernebjerg (May 26 2021 at 11:03):
But as far as I can tell from debugging, the install method being called is the one returning void which does not seem to call fetchAndInstallDependencies (unlike the install method returning PackageInstallOutcomeJson`). Or am missing smt.?
Morten Ernebjerg (May 26 2021 at 11:06):
(the latter install function, on the other hand, expects the StructureDefs to already have snapshots, cf. JDoc)
Jens Villadsen (May 26 2021 at 14:53):
Ok ... so assuming you are using the hapi-fhir-starter-project ... try add the following:
.setFetchDependencies(true)
.addDependencyExclude("hl7.fhir.r4.core")
around line 366, in BaseJpaRestfulServer, then you will see things start happening
Jens Villadsen (May 26 2021 at 14:54):
Eventually you will see the following error:
2021-05-26 16:52:02.919 [main] INFO c.u.f.jpa.packages.JpaPackageCache [JpaPackageCache.java:195] Parsing package .tar.gz (28612 bytes) from http://packages.fhir.org/kbv.basis/1.1.3
2021-05-26 16:52:02.948 [main] ERROR o.a.c.c.C.[Tomcat].[localhost].[/] [DirectJDKLog.java:175] Servlet.init() for servlet [jpaRestfulServer] threw exception
ca.uhn.fhir.rest.server.exceptions.InvalidRequestException: Package ID KBV.Basis doesn't match expected: kbv.basis
at ca.uhn.fhir.jpa.packages.JpaPackageCache.addPackageToCache(JpaPackageCache.java:199)
at ca.uhn.fhir.jpa.packages.JpaPackageCache.loadPackage(JpaPackageCache.java:398)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)
at com.sun.proxy.$Proxy213.loadPackage(Unknown Source)
at ca.uhn.fhir.jpa.packages.PackageInstallerSvcImpl.fetchAndInstallDependencies(PackageInstallerSvcImpl.java:265)
at ca.uhn.fhir.jpa.packages.PackageInstallerSvcImpl.fetchAndInstallDependencies(PackageInstallerSvcImpl.java:268)
Jens Villadsen (May 26 2021 at 14:57):
with those line added you will see that the packages are found and installed recursively and other stuff starts to bug you from the actual packages
Morten Ernebjerg (May 26 2021 at 15:35):
Ah, thanks for the tip! Indeed, I see the same now - looks like a version of the recent issue with upper vs. lower-case in package IDs discussed in this thread. Grahame put out a fix in the validator on May 14th, I'm guessing that hasn't propagated to the JPA starter yet?
Morten Ernebjerg (May 27 2021 at 13:23):
Update: Turns out to be a similar but separate issue, proposed fix here: https://github.com/hapifhir/hapi-fhir/pull/2683
Jens Villadsen (May 28 2021 at 15:33):
Did you just link to your own link ;)
Patrick Werner (May 29 2021 at 15:13):
@Morten Ernebjerg merged! Thanks for the contribution.
Patrick Werner (May 30 2021 at 13:08):
(deleted)
Jens Villadsen (Jun 23 2021 at 09:28):
Jens Villadsen said:
Ok ... so assuming you are using the hapi-fhir-starter-project ... try add the following:
.setFetchDependencies(true) .addDependencyExclude("hl7.fhir.r4.core")around line 366, in BaseJpaRestfulServer, then you will see things start happening
all in favour that I add this as a configuration option for the starter project? @Patrick Werner @Morten Ernebjerg ?
Jens Villadsen (Jun 23 2021 at 09:28):
or should it be default behavior?
Patrick Werner (Jun 23 2021 at 09:30):
I think this should be default behaviour. I also want to add ConceptMaps to the loaded ResourceTypes
Morten Ernebjerg (Jun 23 2021 at 09:30):
@Jens Villadsen THanks for suggesting this - I would actually say it should be the default behavior (that was my ow naive assumption when I started...). Are there situation where one would not want this?
Jens Villadsen (Jun 23 2021 at 10:01):
what are the other packages Id's that should always be ignored (besides the r4)?
Jens Villadsen (Jun 24 2021 at 10:03):
https://github.com/hapifhir/hapi-fhir-jpaserver-starter/pull/254 @Patrick Werner @Morten Ernebjerg @James Agnew
Patrick Werner (Jun 24 2021 at 11:01):
merged. Thanks @Jens Villadsen
Last updated: Apr 12 2022 at 19:14 UTC
