Stream: implementers
Topic: TerminologyCache is the source of much performance pain
Jonathan Meijer (Jan 20 2022 at 16:59):
When validating messages, the validateCode(ValidationOptions options, CodeableConcept code, ValueSet vs) method in org.hl7.fhir.r5.BaseWorkerContext (called by checkCodeOnServer in org.hl7.fhir.validation.instance.InstanceValidator) creates a token to find a cached validation result.
While the purpose of a cache is usually to improve performance, this heavy lookup takes 15-20% of the average validation entire time. This is because TerminologyCache.extracted(...) converts the entire ValueSet into JSON, and the UTF8 conversion takes most of the time.
The issue appears to be in TerminologyCache.extracted(...). Seems that in order to build the cache token, it'd be simply a lot more efficient to use the ValueSet.url instead of using the whole ValueSet?!?
Any comments on the matter?
Jonathan Meijer (Jan 20 2022 at 21:13):
Noticed CacheToken.request is never used except to generate CacheToken.key.
Vassil Peytchev (Jan 21 2022 at 02:47):
Is that regarding the validator code, or IG publisher?
Lloyd McKenzie (Jan 21 2022 at 04:35):
@Grahame Grieve to consider when you're back
Jonathan Meijer (Jan 21 2022 at 13:22):
Vassil Peytchev said:
Is that regarding the validator code, or IG publisher?
Inside the Validator code.
Grahame Grieve (Jan 27 2022 at 05:11):
hmm. I thought I had refactored that code.
Grahame Grieve (Jan 27 2022 at 05:12):
I'll look at it again
Grahame Grieve (Jan 28 2022 at 05:04):
will be improved next release
Grahame Grieve (Jan 28 2022 at 05:04):
though it does change the behavior of the validator somewhat...
Grahame Grieve (Jan 28 2022 at 05:05):
the validator has been robust about breaking the versioning rules - offering up the same value set by url + version that contains different content. From now on, that won't work the same. That was something @James Agnew and I already decided on
Grahame Grieve (Jan 28 2022 at 05:05):
I just didn't finish the change off
Jonathan Meijer (Mar 11 2022 at 18:31):
FYI, average time required to validate doubled from 5.4.11 to 5.6.36. Though there are many cases of Spring.replace which take up a lot of time in processing (through Pattern match), replacing those occurrences with StringUtils.replace/remove do not appear to improve performance much.
Last updated: Apr 12 2022 at 19:14 UTC
