FHIR Chat · validator core questions · implementers

Stream: implementers

Topic: validator core questions

Jens Villadsen (Feb 14 2020 at 12:57):

@Grahame Grieve is this correct? https://github.com/hapifhir/org.hl7.fhir.core/blob/17eb1c41f7300f6e407cb043ed8b27d1ed2b3783/org.hl7.fhir.validation/src/main/java/org/hl7/fhir/r5/validation/InstanceValidator.java#L2755 - as I read https://www.hl7.org/fhir/stu3/http.html#2.21.0.17.2 - (Conditional References), relative references with query parameters in the reference are allowed as long as it is contained in a transaction bundle

Grahame Grieve (Feb 16 2020 at 09:20):

yes

Jens Villadsen (Feb 16 2020 at 19:59):

hmmm ... validating this:

{
  "entry": [
    {
      "request": {
        "ifNoneExist": "url=http://ehealth.sundhed.dk/fhir/SearchParameter/StructureDefinition/url",
        "url": "SearchParameter",
        "method": "POST"
      },
      "resource": {
        "id": "ehealth-structuredefinition-search-url",
        "xpathUsage": "normal",
        "description": "Search parameter for finding a structureDefinition by URL",
        "resourceType": "SearchParameter",
        "code": "url",
        "status": "active",
        "xpath": "f:StructureDefinition/f:url",
        "type": "uri",
        "expression": "StructureDefinition.url",
        "name": "url",
        "base": [
          "StructureDefinition"
        ],
        "url": "http://ehealth.sundhed.dk/fhir/SearchParameter/StructureDefinition/url"
      },
      "fullUrl": "urn:uuid:58441a26-dd41-4e0b-9cd0-6537ecdf69af"
    },
    {
      "request": {
        "ifNoneExist": "url=http://ehealth.sundhed.dk/fhir/SearchParameter/StructureDefinition/url-unique",
        "url": "SearchParameter",
        "method": "POST"
      },
      "resource": {
        "id": "ehealth-structuredefinition-search-url-unique",
        "extension": [
          {
            "valueBoolean": true,
            "url": "http://hapifhir.io/fhir/StructureDefinition/sp-unique"
          }
        ],
        "description": "Search parameter for making a structureDefinition unique by URL",
        "resourceType": "SearchParameter",
        "name": "unique url",
        "status": "active",
        "component": [
          {
            "expression": "StructureDefinition",
            "definition": {
              "reference": "SearchParameter?url=http://ehealth.sundhed.dk/fhir/SearchParameter/StructureDefinition/url"
            }
          }
        ],
        "type": "composite",
        "expression": "StructureDefinition.url",
        "base": [
          "StructureDefinition"
        ],
        "code": "sd-unique-url",
        "url": "http://ehealth.sundhed.dk/fhir/SearchParameter/StructureDefinition/url-unique"
      },
      "fullUrl": "urn:uuid:f0a77d9a-fd10-48ed-85f9-51906cf46d29"
    }
  ],
  "type": "transaction",
  "resourceType": "Bundle"
}

• against HAPI FHIR 4.2.0 (which does not contain the latest release of the validator core release) ends up in this:

{
  "resourceType": "OperationOutcome",
  "text": {
    "status": "generated",
    "div": "<div xmlns=\"http://www.w3.org/1999/xhtml\"><h1>Operation Outcome</h1><table border=\"0\"><tr><td style=\"font-weight: bold;\">INFORMATION</td><td>[Bundle.entry[1].resource.extension[0], Line 36, Col 12]</td><td><pre>Unknown extension http://hapifhir.io/fhir/StructureDefinition/sp-unique</pre></td>\n\t\t\t\t\t\n\t\t\t\t\n\t\t\t</tr>\n\t\t\t<tr>\n\t\t\t\t<td style=\"font-weight: bold;\">ERROR</td>\n\t\t\t\t<td>[Bundle.entry[1].resource.component[0].definition]</td>\n\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\t<td><pre>Relative URLs must be of the format [ResourceName]/[id].  Encountered SearchParameter?url=http://ehealth.sundhed.dk/fhir/SearchParameter/StructureDefinition/url</pre></td>\n\t\t\t\t\t\n\t\t\t\t\n\t\t\t</tr>\n\t\t</table>\n\t</div>"
  },
  "issue": [
    {
      "severity": "information",
      "code": "processing",
      "diagnostics": "Unknown extension http://hapifhir.io/fhir/StructureDefinition/sp-unique",
      "location": [
        "Bundle.entry[1].resource.extension[0]",
        "Line 36, Col 12"
      ]
    },
    {
      "severity": "error",
      "code": "processing",
      "diagnostics": "Relative URLs must be of the format [ResourceName]/[id].  Encountered SearchParameter?url=http://ehealth.sundhed.dk/fhir/SearchParameter/StructureDefinition/url",
      "location": [
        "Bundle.entry[1].resource.component[0].definition"
      ]
    }
  ]
}

• which looks like a bug to me

Paul Church (Feb 16 2020 at 20:04):

That's interesting - the conditional reference would be resolved if the transaction was posted, but should it pass validation? It's not a valid reference if stored in that form.

Jens Villadsen (Feb 16 2020 at 20:05):

pasted image

Jens Villadsen (Feb 16 2020 at 20:05):

from https://www.hl7.org/fhir/stu3/http.html#2.21.0.17.2

Jens Villadsen (Feb 16 2020 at 21:19):

@Paul Church - the rules following that snippet says that the query should be resolved. If resolved, the query is replaced: pasted image

Paul Church (Feb 16 2020 at 21:34):

Well that's my point...it's not a valid resource. It's a valid input to the bundle endpoint, which is transformed according to those rules (resolving the query) to produce a valid resource.

Jens Villadsen (Feb 16 2020 at 22:25):

which is where I'm posting it to

Jens Villadsen (Feb 16 2020 at 22:28):

so if the content in its whole is valid according to spec, how come it does not pass the validation, @Grahame Grieve?

Grahame Grieve (Feb 17 2020 at 01:15):

because no one tested it yet

Grahame Grieve (Feb 17 2020 at 01:46):

this is the errors I get with the current validator:

ERROR: Bundle.entry[1].resource.component[0].definition: This property must be an simple value, not an object
warning: WARNING: Bundle.entry[0].resource: spd-0: Name should be usable as an identifier for the module by machine processing applications such as code generation [name.matches('A-Z{0,254}')]
warning: WARNING: Bundle.entry[1].resource: spd-0: Name should be usable as an identifier for the module by machine processing applications such as code generation [name.matches('A-Z{0,254}')]
ERROR: Bundle.entry[1].resource.extension[0]: The extension http://hapifhir.io/fhir/StructureDefinition/sp-unique is unknown, and not allowed here
ERROR: Bundle.entry[1].resource.component[0]: Profile http://hl7.org/fhir/StructureDefinition/SearchParameter, Element 'Bundle.entry[1].resource.component[0].definition': minimum required = 1, but only found 0

Grahame Grieve (Feb 17 2020 at 01:46):

do you think that any of these errors are wrong?

Grahame Grieve (Feb 17 2020 at 01:47):

oops, no, these errors:

Grahame Grieve (Feb 17 2020 at 01:48):

ERROR: Bundle.entry[1].resource.extension[0]: The extension http://hapifhir.io/fhir/StructureDefinition/sp-unique is unknown, and not allowed here
ERROR: Bundle.entry[1].resource.component[0].definition: Relative URLs must be of the format [ResourceName]/[id]. Encountered SearchParameter?url=http://ehealth.sundhed.dk/fhir/SearchParameter/StructureDefinition/url

Jens Villadsen (Feb 17 2020 at 08:33):

why is the following not allowed?

ERROR: Bundle.entry[1].resource.extension[0]: The extension http://hapifhir.io/fhir/StructureDefinition/sp-unique is unknown, and not allowed here

its a normal extension - did you run the validator with strictness that does not allow unknown extensions?

• and the following:

ERROR: Bundle.entry[1].resource.component[0].definition: Relative URLs must be of the format [ResourceName]/[id]. Encountered SearchParameter?url=http://ehealth.sundhed.dk/fhir/SearchParameter/StructureDefinition/url

• that should be allowed according to the spec, so that should not be an error. Instead such use should be allowed when in a bundle with type transaction. It may not compute runtime, but it is allowed according to spec. The real question is more: Should such search parameter resolve within the bundle or not. That is not stated clearly. We already have the ifNoneExistwhich does the resolving according to the persisted data. My guess would be that such use of search within the reference should resolve within the bundle

Jens Villadsen (Feb 17 2020 at 08:33):

@Grahame Grieve

Grahame Grieve (Feb 17 2020 at 08:35):

The first: by default the validator only allows extensions it knows about. see the validator doco at https://confluence.hl7.org/display/FHIR/Using+the+FHIR+Validator

Grahame Grieve (Feb 17 2020 at 08:35):

I'll look at the second

Jens Villadsen (Feb 17 2020 at 08:36):

regarding "The first" - this is as expected.

Jens Villadsen (Feb 17 2020 at 21:36):

@Grahame Grieve so whats the verdict? should a search be allowed within the bundle and use placeholder ID's that are later resolved? ... or where does it end?

Grahame Grieve (Feb 18 2020 at 03:22):

ok fixed next release

Jens Villadsen (Feb 18 2020 at 06:02):

as in: "The validator now allows it and verifies a search within the bundle" or "This part is now changed in the spec"? Don't this have implications on the reference implementations?

Grahame Grieve (Feb 18 2020 at 06:35):

the validator now allows valid search references for references in a transaction bundle

Jens Villadsen (Feb 18 2020 at 08:00):

@James Agnew in case this is not on your radar ...

Grahame Grieve (Feb 18 2020 at 09:25):

@James Agnew if the validator is more than an hour old or so, it’s too old for someone ;-)

Jens Villadsen (Feb 18 2020 at 10:23):

lol

Jens Villadsen (Feb 18 2020 at 10:24):

@Grahame Grieve - no I was just wondering if HAPI has problems with how this is interpreted

James Agnew (Feb 18 2020 at 13:08):

FWIW, the version of the validator that is in the 4.2.0 release (just released) is about 3 weeks old, as we went into feature freeze a while back in order to shake out bugs for stable.

We've already pushed new 4.3.0-SNAPSHOT builds to Maven though that incorporate the bleeding-edge validator.

Jens Villadsen (Feb 18 2020 at 15:20):

as expected ... ;)

Jens Villadsen (Feb 18 2020 at 15:21):

@James Agnew how stable are the snapshots ? do they pass all automated tests?

James Agnew (Feb 18 2020 at 15:30):

Yup, we push when the tests are passing. We've found the snapshots to be quite stable in the end.. We're pushing them roughly every week.

Jens Villadsen (Feb 18 2020 at 15:33):

ok, we're almost done upgrading to 4.2.0 (still bugtracking why the entity manager is not providing me with a transaction - probably some local stuff), but it could look like we might already need to go up to 4.3.0 snapshot ...

Jens Villadsen (Mar 02 2020 at 09:44):

@Grahame Grieve
Consider the following:

FHIR Validation tool Version 4.2.4-SNAPSHOT (Git# a3d7acdf351d). Built 2020-02-28T15:17:35.66+11:00 (3 days old)
Detected Java version: 11.0.3 from C:\Program Files\AdoptOpenJDK\jre-11.0.3.7-hotspot on amd64 (64bit). 4070MB available
Arguments: source\examples\patient-example-bubber.json -ig source\resources
Directories: Current = C:\Users\Jensk\Documents\dk-core-r4, Package Cache = C:\Users\Jensk\.fhir\packages
  .. FHIR Version current, definitions from hl7.fhir.r5.core#current
  .. connect to tx server @ http://tx.fhir.org
    (vcurrent)
+  .. load IG from source\resources
  .. validate [source\examples\patient-example-bubber.json]
Validation Infrastructure fail validating source\examples\patient-example-bubber.json: Problem processing expression maritalStatus.coding.exists(code = 'P' and system = 'http://terminology.hl7.org/CodeSystem/v3-MaritalStatus') or maritalStatus.coding.exists(code = 'A' and system = 'http://terminology.hl7.org/CodeSystem/v3-MaritalStatus') in profile http://hl7.org/fhir/dk/core/StructureDefinition/dk-core-patient path Patient: Error in ?? at 1, 8: The function "exists" requires 0 parameters
Exception in thread "main" org.hl7.fhir.exceptions.FHIRException: Problem processing expression maritalStatus.coding.exists(code = 'P' and system = 'http://terminology.hl7.org/CodeSystem/v3-MaritalStatus') or maritalStatus.coding.exists(code = 'A' and system = 'http://terminology.hl7.org/CodeSystem/v3-MaritalStatus') in profile http://hl7.org/fhir/dk/core/StructureDefinition/dk-core-patient path Patient: Error in ?? at 1, 8: The function "exists" requires 0 parameters
        at org.hl7.fhir.validation.instance.InstanceValidator.checkInvariant(InstanceValidator.java:5218)
        at org.hl7.fhir.validation.instance.InstanceValidator.checkInvariants(InstanceValidator.java:5203)
        at org.hl7.fhir.validation.instance.InstanceValidator.checkInvariants(InstanceValidator.java:5094)
        at org.hl7.fhir.validation.instance.InstanceValidator.validateElement(InstanceValidator.java:4605)
        at org.hl7.fhir.validation.instance.InstanceValidator.startInner(InstanceValidator.java:3456)
        at org.hl7.fhir.validation.instance.InstanceValidator.start(InstanceValidator.java:3380)
        at org.hl7.fhir.validation.instance.InstanceValidator.validateResource(InstanceValidator.java:5311)
        at org.hl7.fhir.validation.instance.InstanceValidator.validate(InstanceValidator.java:681)
        at org.hl7.fhir.validation.instance.InstanceValidator.validate(InstanceValidator.java:535)
        at org.hl7.fhir.validation.ValidationEngine.validate(ValidationEngine.java:1077)
        at org.hl7.fhir.validation.ValidationEngine.validate(ValidationEngine.java:991)
        at org.hl7.fhir.validation.Validator.main(Validator.java:659)

Shouldn't this use of exists be valid?

Jens Villadsen (Mar 02 2020 at 09:45):

According to http://hl7.org/fhirpath/#existence and the stated example use, I read it as if this kind of use of exists should be valid

Grahame Grieve (Mar 04 2020 at 02:59):

yes, you;re right. Don't know how the fhirpath tests missed that

Grahame Grieve (Mar 04 2020 at 03:25):

fixed next release

Jens Villadsen (Mar 04 2020 at 22:39):

@Grahame Grieve I have an observation and I wonder if it is a bug or a shortcoming in the validator: I have resource profile on Patient. General practitioner is sliced and discriminator is on type=value, path=identifier.system. The identifier has its own datatype resource profile - everything works .... - well sort of. Within the datatype profile on identifier, I have a FHIR path constraint on the identifier.value - but it seems that the validator never evaluates the FHIR path expression when validating the patient resource. Everything else works and I can see that it evaluates the cardinalities stated in the datatype profile on identifier. Is this a bug in the validator or is it as expected?

Lloyd McKenzie (Mar 04 2020 at 22:41):

That sounds like a bug. Submit a narrow example we can use as a test case

Jens Villadsen (Mar 04 2020 at 22:42):

will do ... is my use exotic?

Jens Villadsen (Mar 04 2020 at 23:01):

@Lloyd McKenzie here or on jira?

Lloyd McKenzie (Mar 04 2020 at 23:09):

Jira. And no, not that exotic

Jens Villadsen (Mar 04 2020 at 23:21):

https://jira.hl7.org/browse/FHIR-26440 @Lloyd McKenzie

Grahame Grieve (Mar 04 2020 at 23:37):

this one needs a test case- that is, a valid and an invalid example, and as few profiles as required to reproduce the case. And it should be a gibhub issue not a JIRA task

Jens Villadsen (Mar 04 2020 at 23:44):

@Grahame Grieve - did you have a look at what I submitted? It contains a resource profile and a datatype profile and a resource instance that shows that is does not fail when it ought to

Jens Villadsen (Mar 04 2020 at 23:47):

and now it also contains a resource example that should not fail

Grahame Grieve (Mar 05 2020 at 00:13):

so I don't see why you think there's a problem with the validator here

Grahame Grieve (Mar 05 2020 at 00:17):

no. hang on

Grahame Grieve (Mar 05 2020 at 05:39):

ok fixed next release

Jens Villadsen (Mar 05 2020 at 08:46):

image.png ;)

Grahame Grieve (Mar 05 2020 at 08:49):

isn't lenin cat in the mausoleum in the red square with lenin?

Jens Villadsen (Mar 05 2020 at 08:55):

can't and won't say - it would violate my contract with my client :D

Jens Villadsen (Mar 05 2020 at 08:56):

ETA of next release would be ....? Within a couple of ours?

Grahame Grieve (Mar 05 2020 at 09:03):

it's already done

Jens Villadsen (Mar 05 2020 at 14:03):

@Grahame Grieve I'm pretty sure you broke something else now

Jens Villadsen (Mar 05 2020 at 14:04):

dk-core-episodeOfCare.StructureDefinition.xml episodeOfCare-example.json episodeOfCare-example-wrong.json

Jens Villadsen (Mar 05 2020 at 14:05):

there's a profile and two examples attached. One of the resources should fail the other should pass

Jens Villadsen (Mar 05 2020 at 14:06):

the profile is the simplified version. The complex one is where I slice the identifiers on system and add different profiles for the identfiers. That part is not attached. Get the simple one to work is the first step. It seems the FhirPath invariants are not evaluated

Jens Villadsen (Mar 05 2020 at 14:06):

Running on FHIR Validation tool Version 4.2.6-SNAPSHOT (Git# d8262c056c7b). Built 2020-03-05T16:54:24.93+11:00 (8 hours old)

Jens Villadsen (Mar 05 2020 at 14:08):

wait ....

Jens Villadsen (Mar 05 2020 at 14:10):

nope - error was on my side. I FAILED in declaring the profile in meta ....

Last updated: Apr 12 2022 at 19:14 UTC