Stream: hapi
Topic: supporting not-present code systems in Hapi 3.8.0
Matti Uusitalo (Jun 11 2019 at 11:56):
We have a CodeSystem which refers to a huge external code system which is not practical to implement as a resource in Hapi. Before upgrading to 3.8.0 we could work around it by marking the codesystem as "not-present".
With the new release we are facing problems as Hapi validator reports errors about invalid code sets:
"None of the codes provided are in the value set http://phr.kanta.fi/ValueSet/fiphr-vs-vnrcode (http://phr.kanta.fi/ValueSet/fiphr-vs-vnrcode, and a code from this value set is required) (codes = urn:oid:1.2.246.537.6.55#168120)"
I had a look at the validator internals and there seems to be a possibility to support a ValueSetChecker which looks like it was meant to be used in these cases but I don't see a way to provide such an implementation so that Hapi would use it to validate my CodeableConcept.
How can I provide Hapi with an implementation which makes the validator accept any code and just check the system?
This is an example of how the codeable concept looks like as a resource. The binding strength is REQUIRED as the values must be from the code system bound to the CodeableConcept.
"medicationCodeableConcept":{
"coding":[{
"system": "urn:oid:1.2.246.537.6.55",
"display":"Amoxicillin (substance)",
"code": "168120"
}],
"text": "SOMAC CONTROL"
},
Grahame Grieve (Jun 11 2019 at 14:54):
if the infrastructure doesn't know the code system at all, it will just create a hint saying it doesn't know it and can't validate it. Why not leave it like that?
Patrick Werner (Jun 11 2019 at 14:58):
@Matti Uusitalo edit: re-read your question.
Patrick Werner (Jun 11 2019 at 15:01):
i agree it would be good to at least check if the system was specified correct. You could alter your profile and remove the VS binding and add a fixed value for the system, but i assume this is not what you want.
Patrick Werner (Jun 11 2019 at 15:03):
We have a similar use case with CodeSystems to big to include it into hapi, eventually we will delegate this part of the validation to an external Terminology Server.
For the meantime it would be important to at least check the system value. Just ignore everything seems not the way to go
Grahame Grieve (Jun 11 2019 at 15:33):
you can't check the system value - so it's there, wrong? how do you know it's wrong?
Matti Uusitalo (Jun 12 2019 at 05:32):
if the infrastructure doesn't know the code system at all, it will just create a hint saying it doesn't know it and can't validate it. Why not leave it like that?
We get error level issues from the validator. This wasn't the case with Hapi 3.6.0.
Matti Uusitalo (Jun 12 2019 at 05:42):
i agree it would be good to at least check if the system was specified correct. You could alter your profile and remove the VS binding and add a fixed value for the system, but i assume this is not what you want.
Thanks for the suggestion, that's definitely worth checking out.
Matti Uusitalo (Jun 12 2019 at 06:33):
@Grahame Grieve If I use noTerminologyChecks flag this seems to work. Is that what you meant?
I have to check against our whole test suite before I can be sure, but at least my initial tests are promising.
Matti Uusitalo (Jun 12 2019 at 08:04):
unfortunately we can't configure noTerminologyChecks to be true as it does not validate againts hl7 codesystems then
Patrick Werner (Jun 12 2019 at 08:19):
you can't check the system value - so it's there, wrong? how do you know it's wrong?
i don't know if it is right, but i know if it is wrong (wrong system) . But if the profile has an required VS binding and the system doesn't match we could thrown a error. If the system is matching, but the CS is emtpy or missing it could be a warning.
Eeva Turkka (Jun 12 2019 at 08:33):
This actually used to work like that with the "not-present" value set to the codesystem resource (https://www.hl7.org/fhir/valueset-codesystem-content-mode.html). This is useful in cases when there are two codesystems that can be validated and then there are values from a third one that can't: https://simplifier.net/finnishphr/fiphr-vs-activitycode .
Grahame Grieve (Jun 12 2019 at 14:23):
I Did not mean the noTerminologyChecks Flag. By default, if you use a code system that the validator doesn’t know, it will simply note that and move in. Is that not the right outcome?
Matti Uusitalo (Jun 13 2019 at 05:32):
I Did not mean the noTerminologyChecks Flag. By default, if you use a code system that the validator doesn’t know, it will simply note that and move in. Is that not the right outcome?
That would be good. Unfortunately it is not what happens. Here's the reason, so far as I understand:
Here's the loop which checks if the code is ok
https://github.com/jamesagnew/hapi-fhir/blob/master/hapi-fhir-structures-dstu3/src/main/java/org/hl7/fhir/dstu3/hapi/ctx/HapiWorkerContext.java#L246
Here's the part where the information aobut not present code system is returned
https://github.com/jamesagnew/hapi-fhir/blob/master/hapi-fhir-structures-dstu3/src/main/java/org/hl7/fhir/dstu3/hapi/ctx/HapiWorkerContext.java#L298
Unfortunately a ValidationResult without a ConceptDefinitionComponent returns false from isOk(). The upper loop then ends up returning a validation error.
Grahame Grieve (Jun 13 2019 at 11:57):
oh. you are using a very very old version of the validator. You should be using the latest R5 validator
Grahame Grieve (Jun 13 2019 at 11:57):
in hapi-core
James Agnew (Jun 13 2019 at 16:14):
HAPI FHIR 3.8.0 uses the latest R4 validator (not the R5 one, we're going to make that switch during the current development cycle) but it's not hugely out of date either.
"Not present" doesn't mean "can't validate" to HAPI though. If you load LOINC/SCT/etc into the JPA server for example, you get a not-present CodeSystem because the content is loaded into non-FHIR tables in the JPA server.
You should probably be defining an IValidationSupport implementation that always accepts codes in your code system if you want them to be ignored.
Morten Ernebjerg (Jun 14 2019 at 07:06):
Hi @James Agnew Does having the R4 validator in HAPI 3.8 mean that when validating STU3 resources, one has to first transform the STU3 StructureDefs into R4 format? (this was my, possibly faulty, understanding of an earlier thread that I unfortunately cannot find now). If so, are there available examples for that?
Patrick Werner (Jun 14 2019 at 07:13):
@Morten Ernebjerg this is done by hapi automatically for you.
Morten Ernebjerg (Jun 14 2019 at 07:23):
Thanks, @Patrick Werner !
Matti Uusitalo (Jun 14 2019 at 10:39):
HAPI FHIR 3.8.0 uses the latest R4 validator (not the R5 one, we're going to make that switch during the current development cycle) but it's not hugely out of date either.
"Not present" doesn't mean "can't validate" to HAPI though. If you load LOINC/SCT/etc into the JPA server for example, you get a not-present CodeSystem because the content is loaded into non-FHIR tables in the JPA server.
You should probably be defining an IValidationSupport implementation that always accepts codes in your code system if you want them to be ignored.
I don't fully understand how IValidationSupport can affect it, as HapiWorkerContext doesn't delegate checking codes to IValidationSupport. As far as I can tell it is the same in the R4 version. This is our current workaround to the problem in R3 HapiWorkerContext:
Matti Uusitalo (Jun 20 2019 at 07:45):
I studied it further. We tried to implement a IValidationSupport which returns ok on the validate method. This does not help as Hapi actually validates the code in two places. I debugged the system and took two stacktraces where validateCode is called. Have a look at this diff from the stacktraces:
< HapiWorkerContext.validateCode(String, String, String, ValueSet) line: 297
< FhirInstanceValidator$WorkerContextWrapper.validateCode(String, ValueSet) line: 780
< InstanceValidator.checkPrimitiveBinding(List<ValidationMessage>, String, String, ElementDefinition, Element, StructureDefinition) line: 1681
< InstanceValidator.checkPrimitive(Object, List<ValidationMessage>, String, String, ElementDefinition, Element, StructureDefinition) line: 1599
< InstanceValidator.validateElement(ValidatorHostContext, List<ValidationMessage>, StructureDefinition, ElementDefinition, StructureDefinition, ElementDefinition, Element, Element, String, NodeStack, boolean) line: 3751
PhrValidationSupport.validateCode(FhirContext, String, String, String) line: 179
ValidationSupportChain.validateCode(FhirContext, String, String, String) line: 158
HapiWorkerContext.validateCode(String, String, String) line: 267
FhirInstanceValidator$WorkerContextWrapper.validateCode(String, String, String) line: 748
InstanceValidator.checkCode(List<ValidationMessage>, Element, String, String, String, String) line: 810
InstanceValidator.checkCoding(List<ValidationMessage>, String, Element, StructureDefinition, ElementDefinition, boolean) line: 1051
InstanceValidator.validateElement(ValidatorHostContext, List<ValidationMessage>, StructureDefinition, ElementDefinition, StructureDefinition, ElementDefinition, Element, Element, String, NodeStack, boolean) line: 3758
InstanceValidator.validateElement(ValidatorHostContext, List<ValidationMessage>, StructureDefinition, ElementDefinition, StructureDefinition, ElementDefinition, Element, Element, String, NodeStack, boolean) line: 3843
The first validation is done before our code gets called. Even thoug our IValidationSupport returns a 'ok' result it doesn't matter as the earlier validation rejects the code as non ok.
Grahame Grieve (Jun 20 2019 at 07:46):
are you able to create a test case for this?
Matti Uusitalo (Jun 20 2019 at 10:52):
are you able to create a test case for this?
Certainly
https://github.com/mattiuus/hapi-fhir/commit/456a849447f865dd5e72fc5977835df6ff63e63e
I added tests where the other one simulates our old IValidationSupport implementation and the other one the current.
Last updated: Apr 12 2022 at 19:14 UTC
