FHIR Chat · Simplifier packages and Java Validator

Stream: conformance

Topic: Simplifier packages and Java Validator

Noemi Deppenwiese (Oct 23 2019 at 14:34):

Can I supply Simplifier-generated packages to the validator JAR? I downloaded the package from Simplifier and tried to validate some resources while pointing the -ig Flag to the *.tgz file, but I keep getting

Exception in thread "main" java.lang.NullPointerException
        at org.hl7.fhir.utilities.cache.NpmPackage.fhirVersion(NpmPackage.java:297)
        at org.hl7.fhir.r5.validation.ValidationEngine.loadPackage(ValidationEngine.java:625)
        at org.hl7.fhir.r5.validation.ValidationEngine.loadPackage(ValidationEngine.java:616)
        at org.hl7.fhir.r5.validation.ValidationEngine.loadIgSource(ValidationEngine.java:511)
        at org.hl7.fhir.r5.validation.ValidationEngine.loadIg(ValidationEngine.java:776)
        at org.hl7.fhir.r5.validation.Validator.main(Validator.java:481)

Tried it with my project (https://simplifier.net/packages/bbmri.de/0.3.2) and one from HL7 Germany (https://simplifier.net/packages/de.basisprofil.r4/0.9.0), resulting in the same error.

Lloyd McKenzie (Oct 23 2019 at 15:37):

@Grahame Grieve

Ward Weistra (Oct 23 2019 at 15:37):

Thanks for trying Noemi, let's debug this together with @Martijn Harthoorn and @Grahame Grieve.

I get the same error validating this example (https://simplifier.net/basisprofil-de-r4/example-patient-de-basis-kvid) against this package (https://simplifier.net/packages/de.basisprofil.r4/0.9.0):

PS C:\Users\Ward\Workspace\java-validator> java -Xmx1024m -jar .\org.hl7.fhir.validator.jar .\Example-patient-de-basis-k
vid.json -ig .\de.basisprofil.r4-0.9.0.tgz -version 4.0.0
FHIR Validation tool Version 4.0.26-SNAPSHOT - Built 2019-10-18T22:28:38.796+11:00 - Git f9b52109cffb
Detected Java version: 1.8.0_231 from C:\Program Files (x86)\Java\jre1.8.0_231 on x86 (32bit). 989MB available
Arguments: .\Example-patient-de-basis-kvid.json -ig .\de.basisprofil.r4-0.9.0.tgz -version 4.0.0
Directories: Current = C:\Users\Ward\Workspace\java-validator, Package Cache = C:\Users\Ward\.fhir\packages
  .. connect to tx server @ http://tx.fhir.org
  .. definitions from hl7.fhir.core#4.0.0
    (v4.0.0)
+  .. load IG from .\de.basisprofil.r4-0.9.0.tgz
Loading .\de.basisprofil.r4-0.9.0.tgz to the package cache
  Fetching:.. done.
Exception in thread "main" java.lang.NullPointerException
        at org.hl7.fhir.utilities.cache.NpmPackage.fhirVersion(NpmPackage.java:297)
        at org.hl7.fhir.r5.validation.ValidationEngine.loadPackage(ValidationEngine.java:625)
        at org.hl7.fhir.r5.validation.ValidationEngine.loadPackage(ValidationEngine.java:616)
        at org.hl7.fhir.r5.validation.ValidationEngine.loadIgSource(ValidationEngine.java:511)
        at org.hl7.fhir.r5.validation.ValidationEngine.loadIg(ValidationEngine.java:776)
        at org.hl7.fhir.r5.validation.Validator.main(Validator.java:481)

And I don't get that error validating this example (http://hl7.org/fhir/us/core/Patient-example.json.html) against the package downloaded from here (http://hl7.org/fhir/us/core/downloads.html):

PS C:\Users\Ward\Workspace\java-validator> java -Xmx1024m -jar .\org.hl7.fhir.validator.jar .\Patient-example.json -ig .
\package-us-core.tgz -version 4.0.0
FHIR Validation tool Version 4.0.26-SNAPSHOT - Built 2019-10-18T22:28:38.796+11:00 - Git f9b52109cffb
Detected Java version: 1.8.0_231 from C:\Program Files (x86)\Java\jre1.8.0_231 on x86 (32bit). 989MB available
Arguments: .\Patient-example.json -ig .\package-us-core.tgz -version 4.0.0
Directories: Current = C:\Users\Ward\Workspace\java-validator, Package Cache = C:\Users\Ward\.fhir\packages
  .. connect to tx server @ http://tx.fhir.org
  .. definitions from hl7.fhir.core#4.0.0
    (v4.0.0)
+  .. load IG from .\package-us-core.tgz
Loading .\package-us-core.tgz to the package cache
  Fetching:............ done.
  .. validate [.\Patient-example.json]
Success...validating .\Patient-example.json:  error:0 warn:0 info:0

It's likely because the package spec is only solidifying now and Grahame and Martijn just recently reached agreement on it: https://wiki.hl7.org/FHIR_NPM_Package_Spec

Ward Weistra (Oct 23 2019 at 15:44):

The hl7.org US-Core package manifest (package/package.json):

{
  "tools-version": 3,
  "type": "fhir.ig",
  "license": "CC0-1.0",
  "title": "US Core R4",
  "dependencies": {
    "hl7.fhir.core": "4.0.0"
  },
  "author": "HL7 International - US Realm Steering Committee",
  "version": "3.0.0",
  "name": "hl7.fhir.us.core",
  "url": "http://hl7.org/fhir/us/core/STU3",
  "canonical": "http://hl7.org/fhir/us/core"
}

The Simplifier package manifest:

{
  "name": "de.basisprofil.r4",
  "version": "0.9.0",
  "description": "Projekt Basisprofilierung R4 (HL7 Deutschland e.V.)",
  "devDependencies": null,
  "canonicals": null,
  "dependencies": {
    "simplifier.core.r4": "4.0.1"
  }
}

Perhaps the Java validator is missing the mandatory author element there? Or relying on other elements, currently marked optional or not listed, like type? @Grahame Grieve

Ward Weistra (Oct 23 2019 at 15:55):

Discrepancies between spec and what I see here:

hl7.org misses description
Simplifier misses author
Simplifier uses canonicals instead of canonical
Simplifier fills description with what hl7.org fills title with?
Both use other elements, but that is allowed (hl7.org: tools-version, type; Simplifier: devDependencies)

Grahame Grieve (Oct 23 2019 at 17:42):

and the version is different.... 4.0.1 does not yet exist

Grahame Grieve (Oct 23 2019 at 17:42):

I'll investigate

Ward Weistra (Oct 24 2019 at 07:59):

Adding author or type to the Simplifier package.json didn't resolve the issue.

Grahame Grieve (Oct 24 2019 at 20:08):

this is because of the content in the package.json:

Grahame Grieve (Oct 24 2019 at 20:09):

  "dependencies": {
    "simplifier.core.r4": "4.0.1"
  }

Grahame Grieve (Oct 24 2019 at 20:09):

The validator doesn't know simplifier.core.r4 and I think it's not valid. nor is 4.0.1 valid for a core version either

Grahame Grieve (Oct 24 2019 at 20:11):

The next version will return a coherent error message in this case

Ward Weistra (Oct 25 2019 at 07:43):

The entire simplifier.core.r4 project, and it's 4.0.1 version, is just our mirroring until we have the official packages on there. Which will be soon, luckily!
Where does the Java validator look for it's dependency packages? Because even if I install the mentioned package in my FHIR package cache with Torinox I run into the same error.

Noemi Deppenwiese (Oct 25 2019 at 13:14):

For a temporary fix, I tried manually editing the package.json on Simplifier:

{
  "name": "BBMRI.de",
  "version": "0.3.3",
  "description": "Profiles for the BBMRI.de / GBA biobanking project.",
  "devDependencies": null,
  "canonicals": null,
  "dependencies": {
    "de.basisprofil.r4": "0.9.0",
    "hl7.fhir.core": "4.0.0"
  }
}

Now, the Validator seems to be able to load the package, but no content (profiles/valuesets) is found:

FHIR Validation tool Version 4.0.26-SNAPSHOT - Built 2019-10-18T22:28:38.796+11:00 - Git f9b52109cffb
[...]
+  .. load IG from GBA\bbmri.de-0.3.3.tgz
Loading GBA\bbmri.de-0.3.3.tgz to the package cache
  Fetching:.. done.
  .. validate [GBA\bbmri-fhir-ig\examples]
Validate GBA\bbmri-fhir-ig\examples\exampleBiobank.json
Terminology server: Check for supported code systems for https://fhir.bbmri.de/CodeSystem/QualityStandard
[...]
Success...validating GBA\bbmri-fhir-ig\examples\exampleBiobank.json:  error:0 warn:2 info:4
  Information @ Organization.extension[0] (line 14, col6) : Unknown extension
[...]
  Warning @ Organization.meta.profile[0] (line 1, col2) : StructureDefinition reference "https://fhir.bbmri.de/StructureDefinition/Biobank" could not be resolved
[...]

Grahame Grieve (Oct 25 2019 at 19:55):

the java validator uses the package cache specification as published. however the reference to hl7.fhir.core is checked directly since that's special - hard coded dependency

Grahame Grieve (Oct 25 2019 at 20:11):

I would not expect manually editing the package to work like that

Ward Weistra (Nov 08 2019 at 11:32):

@Noemi Deppenwiese I assume this is still an issue, right?
@Grahame Grieve Can you see what is wrong with this package that would make the Java validator not find the StructureDefinitions in it? I still get the above errors:

PS C:\Users\Ward\Workspace\java-validator> java -Xmx1024m -jar .\org.hl7.fhir.validator.jar .\exampleBiobank.json -version 4.0.1 -ig .\bbmri.de-0.3.3.tgz
FHIR Validation tool Version 4.0.31-SNAPSHOT - Built 2019-11-05T15:35:39.648+11:00 - Git a0ece162e77c
Detected Java version: 1.8.0_231 from C:\Program Files (x86)\Java\jre1.8.0_231 on x86 (32bit). 989MB available
Arguments: .\exampleBiobank.json -version 4.0.1 -ig .\bbmri.de-0.3.3.tgz
Directories: Current = C:\Users\Ward\Workspace\java-validator, Package Cache = C:\Users\Ward\.fhir\packages
  .. connect to tx server @ http://tx.fhir.org
  .. definitions from hl7.fhir.r4.core#4.0.1
Installing hl7.fhir.r4.core#4.0.1 to the package cache
  Fetching:............................................................................................................
  .............................................................................................|....................................................................................................
  Analysing........................................................................................... done.
    (v4.0.1)
+  .. load IG from .\bbmri.de-0.3.3.tgz
  .. validate [.\exampleBiobank.json]
Terminology server: Check for supported code systems for https://fhir.bbmri.de/CodeSystem/QualityStandard
Success...validating .\exampleBiobank.json:  error:0 warn:2 info:4
  Information @ Organization.extension[0] (line 14, col6) : Unknown extension https://fhir.bbmri.de/StructureDefinition/OrganizationDescription
  Information @ Organization.extension[1] (line 18, col6) : Unknown extension https://fhir.bbmri.de/StructureDefinition/JuridicalPerson
  Information @ Organization.extension[2] (line 22, col6) : Unknown extension https://fhir.bbmri.de/StructureDefinition/QualityStandard
  Information @ Organization.extension[3] (line 34, col6) : Unknown extension https://fhir.bbmri.de/StructureDefinition/QualityStandard
  Warning @ Organization.contact[1].purpose (line 105, col20) : None of the codes provided are in the value set http://hl7.org/fhir/ValueSet/contactentity-type (http://hl7.org/fhir/ValueSet/contactentity-type, and a code should come from this value set unless it has no suitable code) (codes = https://fhir.bbmri.de/CodeSystem/ContactType#RESEARCH)
  Warning @ Organization.meta.profile[0] (line 1, col2) : StructureDefinition reference "https://fhir.bbmri.de/StructureDefinition/Biobank" could not be resolved

Validating with Torinox I only get errors related to the terminology server:

PS C:\Users\Ward\Workspace\java-validator> fhir4 install BBMRI.de 0.3.2
Installed package: BBMRI.de 0.3.2
Updated: package.json
Installed package: de.basisprofil.r4 0.9.0
Restore completed. Saved 'fhirpkg.lock.json'.
PS C:\Users\Ward\Workspace\java-validator> fhir4 validate .\exampleBiobank.json --terminologyserver http://tx.fhir.org/r4
Validating Organization/exampleBiobank
Overall result: SUCCESS

[WARNING] Terminology service failed while validating code 'generated' (system ''): Operation was unsuccessful because of a client error (NotFound). OperationOutcome: <div xmlns="http://www.w3.org/1999/xhtml"><p>Unknown message Unknown ValueSet url: http://hl7.org/fhir/ValueSet/narrative-status|4.0.0</p></div>. (at Organization.text[0].status[0])
[WARNING] Terminology service failed while validating code 'url' (system ''): Operation was unsuccessful because of a client error (NotFound). OperationOutcome: <div xmlns="http://www.w3.org/1999/xhtml"><p>Unknown message Unknown ValueSet url: http://hl7.org/fhir/ValueSet/contact-point-system|4.0.0</p></div>. (at Organization.telecom[0].system[0])
[WARNING] Terminology service failed while validating code 'phone' (system ''): Operation was unsuccessful because of a client error (NotFound). OperationOutcome: <div xmlns="http://www.w3.org/1999/xhtml"><p>Unknown message Unknown ValueSet url: http://hl7.org/fhir/ValueSet/contact-point-system|4.0.0</p></div>. (at Organization.contact[0].telecom[0].system[0])
[WARNING] Terminology service failed while validating code 'phone' (system ''): Operation was unsuccessful because of a client error (NotFound). OperationOutcome: <div xmlns="http://www.w3.org/1999/xhtml"><p>Unknown message Unknown ValueSet url: http://hl7.org/fhir/ValueSet/contact-point-system|4.0.0</p></div>. (at Organization.contact[1].telecom[0].system[0])
[WARNING] Terminology service failed while validating code 'email' (system ''): Operation was unsuccessful because of a client error (NotFound). OperationOutcome: <div xmlns="http://www.w3.org/1999/xhtml"><p>Unknown message Unknown ValueSet url: http://hl7.org/fhir/ValueSet/contact-point-system|4.0.0</p></div>. (at Organization.contact[1].telecom[1].system[0])

(Using package version 0.3.2 because that still had the Simplifier mirror of FHIR Core)

Noemi Deppenwiese (Nov 08 2019 at 12:07):

Yes, I get the same errors for your first example. I published a new version 0.3.4 with a reset of the package manifest, for that I get a slightly improved version of my initial error:

C:\Users\deppenni\GBA\valspace>java -jar validator.jar -version 4.0.0 -ig bbmri.de-0.3.4.tgz resources
FHIR Validation tool Version 4.0.31-SNAPSHOT - Built 2019-11-05T15:35:39.648+11:00 - Git a0ece162e77c
Detected Java version: 1.8.0_212 from C:\Program Files\Java\Amazon Coretto\jdk1.8.0_212\jre on amd64 (64bit). 3612MB available
Arguments: -version 4.0.0 -ig bbmri.de-0.3.4.tgz resources
Directories: Current = C:\Users\deppenni\GBA\valspace, Package Cache = C:\Users\deppenni\.fhir\packages
  .. connect to tx server @ http://tx.fhir.org
  .. definitions from hl7.fhir.r4.core#4.0.1
    (v4.0.1)
+  .. load IG from bbmri.de-0.3.4.tgz
Exception in thread "main" org.hl7.fhir.exceptions.FHIRException: no core dependency or FHIR Version found in the Package definition
        at org.hl7.fhir.utilities.cache.NpmPackage.fhirVersion(NpmPackage.java:364)

Ward Weistra (Nov 08 2019 at 13:53):

@Noemi Deppenwiese Yes, Grahame confirmed that that (having the dependency onsimplifier.core.r4, the Simplifier mirror of the core package) is indeed not expected to work with the Java validator, since the Java validator checks for that package in a different way then for other packages.

Soon Simplifier will not have to mirror those core packages anymore, since we're hard at work to get them one and the same. Soon also within Simplifier the package name for the core will just be hl7.fhir.core
However, I'm also still confused why your workaround (as in package 0.3.3) didn't work for the Java validator. I hope @Grahame Grieve can help us out there, since I expect that issue to persist even when the package names are the same.

Grahame Grieve (Nov 10 2019 at 09:41):

Soon also within Simplifier the package name for the core will just be hl7.fhir.core

Umm, no. use hl7.fhir.r{v}.core. @Ward Weistra

Grahame Grieve (Nov 10 2019 at 09:41):

as for why 0.3.4 fails: here is the package file:

Grahame Grieve (Nov 10 2019 at 09:42):

{
  "name": "BBMRI.de",
  "version": "0.3.4",
  "description": "Profiles for the BBMRI.de / GBA biobanking project.",
  "author": "noemideppenwiese",
  "devDependencies": null,
  "canonicals": null,
  "dependencies": {
    "simplifier.core.r4": "4.0.1"
  }
}

Ward Weistra (Nov 10 2019 at 09:44):

The question is why 0.3.3 fails @Grahame Grieve

Ward Weistra (Nov 10 2019 at 09:45):

Umm, no. use hl7.fhir.r{v}.core. Ward Weistra

Right

Grahame Grieve (Nov 10 2019 at 09:46):

but since i was debugging it... I added support for simplifier.core.rX as a dependency

Grahame Grieve (Nov 10 2019 at 09:47):

but... how did 0.3.3 fail? Looking back through the thread, I don't see any specific information about how it failed?

Grahame Grieve (Nov 10 2019 at 09:51):

but probably it's because the package does not follow the naming convention nor does it have a .index.json

Grahame Grieve (Nov 10 2019 at 09:51):

so all the files in the package are ignored

Grahame Grieve (Nov 10 2019 at 09:59):

but I can fix that, so next version, the validator will just build the .index.json file on the fly when it reads the package

Ward Weistra (Nov 11 2019 at 16:12):

That would be perfect, @Grahame Grieve. And we will add .index.json to our packages soon.

@Noemi Deppenwiese So validating a BBMRI.de or de.basisprofil.r4 (or any Simplifier package) with the Java validator waits for either those packages getting a .index.json (Firely side) or Java validator creating its own (Grahame's side), something we both likely need to do anyway. Likely Grahame will beat us to it ;)
The reference to simplifier.core.r4 shouldn't be a problem anymore with Grahame's latest update.

The validation with Torinox should work already work (as I shared above), as Torinox already does create it's own .index.json:

Install Torinox
fhir4 install BBMRI.de 0.3.4 -- Installs your package and its dependencies, it will automatically get them from Simplifier
fhir4 validate .\theFileYouWantToValidate.json --terminologyserver http://tx.fhir.org/r4

Please let me know if that works for you in the meantime. And I'll keep you posted when we've implemented our part.

The terminology server can't find 4.0.0 valuesets, I assume because Grahame has replaced them with the 4.0.1 version instead.

Grahame Grieve (Nov 11 2019 at 19:23):

hmm

Noemi Deppenwiese (Nov 12 2019 at 11:39):

Thanks for the help! I will investigate if using Torinox for Validation is an interim alternative for our project members. (Currently, we advise them to supply all profiles ect. to the validator jar by placing them in a folder and pointing to it via -ig. This of course requires managing dependencies manually which is why our project was looking to switch to the package system in the first place.)

Grahame Grieve (Nov 12 2019 at 19:46):

this should be all good sometime today

Kevin Mayfield (Jan 01 2020 at 06:34):

If I'm reading this correctly, simplifier packages should work with HL7 validator and probably with hapi fhir?

Grahame Grieve (Jan 01 2020 at 07:18):

SHOULD. that's not the same as 'do' - there's some issues that I'm still working on clarifying

Ward Weistra (Jan 06 2020 at 12:16):

@Kevin Mayfield Please let us know if you run into specific problems so we can resolve them!

Kevin Mayfield (Jan 06 2020 at 12:29):

Yep will do, unfortunately the system I'm working on has to use STU3 and haven't got the resources to run tests on this R4 spec.

Last updated: Apr 12 2022 at 19:14 UTC

Main menu

FHIR Chat · Simplifier packages and Java Validator · conformance

Stream: conformance

Topic: Simplifier packages and Java Validator

Noemi Deppenwiese (Oct 23 2019 at 14:34):

Lloyd McKenzie (Oct 23 2019 at 15:37):

Ward Weistra (Oct 23 2019 at 15:37):

Ward Weistra (Oct 23 2019 at 15:44):

Ward Weistra (Oct 23 2019 at 15:55):

Grahame Grieve (Oct 23 2019 at 17:42):

Grahame Grieve (Oct 23 2019 at 17:42):

Ward Weistra (Oct 24 2019 at 07:59):

Grahame Grieve (Oct 24 2019 at 20:08):

Grahame Grieve (Oct 24 2019 at 20:09):

Grahame Grieve (Oct 24 2019 at 20:09):

Grahame Grieve (Oct 24 2019 at 20:11):

Ward Weistra (Oct 25 2019 at 07:43):

Noemi Deppenwiese (Oct 25 2019 at 13:14):

Grahame Grieve (Oct 25 2019 at 19:55):

Grahame Grieve (Oct 25 2019 at 20:11):

Ward Weistra (Nov 08 2019 at 11:32):

Noemi Deppenwiese (Nov 08 2019 at 12:07):

Ward Weistra (Nov 08 2019 at 13:53):

Grahame Grieve (Nov 10 2019 at 09:41):

Grahame Grieve (Nov 10 2019 at 09:41):

Grahame Grieve (Nov 10 2019 at 09:42):

Ward Weistra (Nov 10 2019 at 09:44):

Ward Weistra (Nov 10 2019 at 09:45):

Grahame Grieve (Nov 10 2019 at 09:46):

Grahame Grieve (Nov 10 2019 at 09:47):

Grahame Grieve (Nov 10 2019 at 09:51):

Grahame Grieve (Nov 10 2019 at 09:51):

Grahame Grieve (Nov 10 2019 at 09:59):

Ward Weistra (Nov 11 2019 at 16:12):

Grahame Grieve (Nov 11 2019 at 19:23):

Noemi Deppenwiese (Nov 12 2019 at 11:39):

Grahame Grieve (Nov 12 2019 at 19:46):

Kevin Mayfield (Jan 01 2020 at 06:34):

Grahame Grieve (Jan 01 2020 at 07:18):

Ward Weistra (Jan 06 2020 at 12:16):

Kevin Mayfield (Jan 06 2020 at 12:29):