Stream: hapi
Topic: FHIR Validation plus Terminology Server
Kevin Mayfield (Oct 31 2019 at 09:40):
I've had FHIR Validation plugged into our server for a while.
Also linked in SNOMED validation using IValidationSupport interface.
I'm not sure if this is the best way of doing this as it often fails (such as trying to expand large ValueSets - the Onto server says the query is too expensive). Is implementing the interface the best way of connecting to a terminology server?
Grahame Grieve (Oct 31 2019 at 10:37):
you should not be doing expand when validating. The validator will call $validate-code
Kevin Mayfield (Oct 31 2019 at 10:41):
I think that's what I just realised.
That implies the ValueSet should be on the terminology server - is that correct?
Grahame Grieve (Oct 31 2019 at 10:41):
you can post it to the server when asking for $validate-code
Michael Lawley (Nov 04 2019 at 00:29):
We have a HAPI server configured to sent all terminology operations to an external Ontoserver instance (Read, Search, Create, etc) as well as hooking in to validation through IValidationSupport. It works quite well.
Kevin Mayfield (Nov 04 2019 at 14:09):
Still getting stuck with expensive ecl statements expansions (which onto won't return)
Codes get validated against this function
@Override
public CodeValidationResult validateCode(FhirContext theContext, String theCodeSystem, String
theCode, String theDisplay)
but the instanceValidator always asks for the ValueSet to be expanded by
@Override
public CodeValidationResult validateCode(FhirContext theContext, String theCodeSystem, String
theCode, String theDisplay)
Jim Steel (Nov 04 2019 at 22:46):
The default too costly threshold for ontoserver is 50000 results. If you need more, the server can be configured to allow more. Are you expecting that many?
Michael Lawley (Nov 04 2019 at 23:07):
And yes, that expand call from the Hapi validation stuff is a real pain.
Grahame Grieve (Nov 05 2019 at 03:47):
@James Agnew I don't know why it calls expand here. It shouldn't
James Agnew (Nov 05 2019 at 10:57):
This logic has been tweaked as of the current master, due out in 13 days. There are 3 cases for code validation:
1- We're validating a code with no valueset binding (as in the case of calling $validate-code)
2- We're validating a code with a VS binding where the VS has a URL (i.e. most validations of FHIR resources with coded fields)
3- We're validating a code with a VS binding where the VS has no URL (say, a contained VS within a Questionnaire definition- I've found this to be surprisingly common)
We never expanded for 1 since there was nothing to expand, but the expansion was there to satisfy 3. I always figured that we cached the results of the validation, so the expansion was a one-time pain. But yeah, I realize now that this is still a pain when you have big expansions.
The new logic only expands on 3, but instead delegates the validation to the term svc for 2. You can see this here: https://github.com/jamesagnew/hapi-fhir/blob/master/hapi-fhir-structures-r4/src/main/java/org/hl7/fhir/r4/hapi/ctx/HapiWorkerContext.java#L247
Michael Lawley (Nov 05 2019 at 12:03):
Hi @James Agnew , there's bugs in there I can see straight off - first case in the method looks for the code explicitly mentioned in the compose.include for a quick win, but fails to account for possible compose.exclude cases.
Michael Lawley (Nov 05 2019 at 12:07):
If you must use expansion logic to handle this (your case 3), then I would suggest crafting an on-the-fly VS that composes the code with an include of the VS under examination. The resulting expansion will have either 0 or 1 members.
James Agnew (Nov 05 2019 at 12:20):
Ah right, the block at the top. That will actually be gone by the 4.1.0 release, it's already removed in a working branch we have going because the valueset pre-expansion makes it obsolete. Agree that approach is too naive.
James Agnew (Nov 05 2019 at 12:20):
good idea about the custom valueset to handle case 3
Grahame Grieve (Nov 05 2019 at 18:23):
I don’t understand why you need expansion in case #3 anyway
James Agnew (Nov 06 2019 at 10:40):
How else would you test whether a code is in the VS?
Grahame Grieve (Nov 06 2019 at 10:45):
use $validate-code
James Agnew (Nov 06 2019 at 10:50):
I don't understand... wouldn't that just validate that the code itself is valid while not actually checking whether it's in the valueset?
The scenario here is things like Questionnaires with a contained VS pointed to by an answerValueSet reference.. So just knowing that the code exists isn't good enough
Kevin Mayfield (Nov 06 2019 at 11:33):
Example call using $validate-code https://ontoserver.dataproducts.nhs.uk/fhir/ValueSet/$validate-code?system=http://snomed.info/sct&code=122298005&url=https://fhir.nhs.uk/STU3/ValueSet/DCH-FinalDeliveryType-1
Kevin Mayfield (Nov 06 2019 at 11:34):
The url specifies the ValueSet to use.
Grahame Grieve (Nov 06 2019 at 11:43):
you can pass the value set inline, rather than referring to it by url. So it doesn't matter that it's an inline anonymous value set.
James Agnew (Nov 06 2019 at 12:19):
If you pass it inline to that operation, how would the term service test membership without expanding the valueset?
Kevin Mayfield (Nov 06 2019 at 16:04):
Thanks @James Agnew 4.1.0 is working like a charm.
Kevin Mayfield (Nov 06 2019 at 16:07):
However it's now revealed quite a lot of new errors in UK Implementation Guide examples :)
Michael Lawley (Nov 06 2019 at 22:13):
You can POST a (Parameters containing a) ValueSet to https://ontoserver.dataproducts.nhs.uk/fhir/ValueSet/$validate-code and we do the validate without "expanding" (equivalent of what I described above)
Grahame Grieve (Nov 06 2019 at 22:39):
it's up the terminology service. But my implementation I have a different path for validate-code that checks the definitions directly
Kevin Mayfield (Nov 07 2019 at 07:50):
@Michael Lawley I'll try that out. What I've done is check the onto server for a UK CodeSystem or ValueSet, if missing add it.
Are their any tools for visualising a SNOMED ValueSet. I can work out what the filters mean but most people won't (don't believe understanding of SNOMED is that high), they would expect to see an expansion but that's not practical in a number of cases.
Michael Lawley (Nov 07 2019 at 08:09):
We have a new version of Shrimp coming soon that provides visualisation of ValueSets by generalising the existing CodeSystem UI to arbitrary ValueSets on the server
Last updated: Apr 12 2022 at 19:14 UTC
