Stream: terminology
Topic: Search all Valuesets
Grahame Grieve (Oct 17 2021 at 21:11):
FHIR#19442 proposes to remove the expand across value sets. It wasn't my favourite piece of API but I'm stunned that the requirement it represents ("text search across all terminology on the server") isn't a compelling thing for other servers.
@Michael Lawley surely your customers are asking for this?
Michael Lawley (Oct 17 2021 at 21:25):
The way the @all valuset is defined (all ValueSets known to the system) is problematic from my perspective - I would be more open to all CodeSystems known to the system (but how to treat multiple versions of the same CodeSystem, especially if additional parameters like property=parent are supplied?)
Michael Lawley (Oct 17 2021 at 21:27):
So far, we've only had one customer ask about this capability.
Michael Lawley (Oct 17 2021 at 21:29):
This part of the definition: "Note that this URL has no fixed meaning - its interpretation is server specific" would, I guess, allow us to provide _some_ support, but I'm not sure how useful it would be in practice.
Grahame Grieve (Oct 17 2021 at 22:16):
I'd be ok with searching all code systems; that was defined before we had CodeSystem.
Grahame Grieve (Oct 17 2021 at 22:17):
I guess that would the GET [base]/CodeSystem/$search?text=xxxxx
Grahame Grieve (Oct 17 2021 at 22:17):
I'm happy to retire @all if we define something equivalent on codesystem
Michael Lawley (Oct 17 2021 at 22:32):
Heh, I had guessed that it was defined before CodeSystem.
If it were instead defined in terms of "all CodeSystems known to the system", we're still stuck with the property problem (although that is not limited to this use-case; I can always define an explicit ValueSet that uses multiple versions of the same CodeSystem).
Peter Jordan (Oct 17 2021 at 22:44):
I agree that this makes more sense for code systems than value sets, but still quite challenging for TS providers to implement. For example, what about searching for incomplete terms? That might easily result in GIGO. It's a feature that I might use, but I'm not sure that I would provide it to anyone entering data in an EHR/EMR/PHR.
Grahame Grieve (Oct 17 2021 at 22:53):
totally wouldn't provide to end-users, but totally important for people building value sets and concept maps
Michael Lawley (Oct 17 2021 at 23:16):
Naive implementation is to just do equivalent of [base]/CodeSystem/_search, construct a ValueSet from the results, then pass through to $expand.
Grahame Grieve (Oct 17 2021 at 23:24):
that's possible, yes. though not many systems support _text on code system, and not on code systems on snomed ct
Michael Lawley (Oct 18 2021 at 00:53):
sorry - I wasn't proposing using _text -- just a way of implementing an implicit ValueSet for all codesystems -- so a search to get "all known code systems, possibly mix in some extras like SNOMED that may only have an implied existence, and build an internal ValueSet set with (long) list of includes.
Grahame Grieve (Oct 18 2021 at 00:55):
sure. but what should the API look like for that? that's what I was proposing as an alternative for getting rid of the all valuesets thign
Michael Lawley (Oct 18 2021 at 01:27):
oh, ok, then you'd want it to return Codings?
Grahame Grieve (Oct 18 2021 at 01:36):
returning an expansion is ok
Michael Lawley (Oct 18 2021 at 08:52):
I think I'd be happier with the implicit ValueSet approach -- my quick prototyping gives reasonable results if I define @all as all codes from the latest (CodeSystem.version) of all CodeSystems known to the system.
Grahame Grieve (Oct 18 2021 at 09:29):
well, I think that a text search of all terminology content is a useful thing to have from a consumer's point of view. So I'm not on board with simply removing the functionality per the task at hand.
Michael Lawley (Oct 19 2021 at 00:27):
Given that "all ValueSets known to the system" should include all the "all codes in a codesystem" implicit valuesets and this would subsume any sub-setting valueset, then I think the @all valueset works as being defined in terms of code systems, not valuesets.
The next question is whether its "latest versions of" or "all versions of" the codesystems.
Michael Lawley (Oct 19 2021 at 00:31):
Doing curl 'http://localhost:8090/fhir/ValueSet/$expand?url=http://hl7.org/fhir/ValueSet/@all&count=10&activeOnly=true&filter=lung' in my prototype with 1246 CodeSystems (mostly FHIR, but several SNOMEDs and LOINC) takes about 8-10 s, finds ~7000 matches and produces 96k of JSON which is mostly just listing all the code system-version pairs scanned.
Michael Lawley (Oct 19 2021 at 00:32):
The contains part of the result looks like:
{
"contains": [
{
"system": "http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl",
"version": "19.11d",
"code": "C12468",
"display": "Lung"
},
{
"system": "http://jim",
"version": "0.0.3",
"code": "lung",
"display": "lung"
},
{
"system": "http://id.who.int/icd11/foundation",
"version": "2019-04",
"code": "1338477196",
"display": "Lung"
},
{
"system": "http://terminology.hl7.org/CodeSystem/v2-0550",
"version": "2.9",
"code": "LUNG",
"display": "Lung"
},
{
"system": "http://jim3",
"version": "0.1.0",
"code": "lung",
"display": "lung"
},
{
"system": "http://terminology.hl7.org/CodeSystem/service-type",
"version": "3.5.0",
"code": "380",
"display": "Lung"
},
{
"system": "http://id.who.int/icd11/mms",
"version": "2019-04",
"code": "XA57M6",
"display": "Lung"
},
{
"system": "http://jim",
"version": "0.0.9",
"code": "lung",
"display": "lung"
},
{
"system": "http://snomed.info/sct",
"version": "http://snomed.info/sct/32506021000036107/version/20200430",
"code": "228220003",
"display": "Lungi"
},
{
"system": "http://snomed.info/sct",
"version": "http://snomed.info/sct/32506021000036107/version/20200531",
"code": "228220003",
"display": "Lungi"
}
]
}
Note that there are separate entries for different versions of the same CodeSystem.
Grahame Grieve (Oct 19 2021 at 02:28):
well... server discretion on that?
Michael Lawley (Oct 20 2021 at 03:18):
My experimental support for http://hl7.org/fhir/ValueSet/@all is now testable on https://r4.ontoserver.csiro.au/fhir
Note that there's > 11M (versioned) codes in this sandbox server, so it will not be doing "realtime" $expand?filter= :-)
Last updated: Apr 12 2022 at 19:14 UTC
