Stream: IG creation
Topic: tx server ignoring $expand version and activeOnly/inactive
Rob Hausam (Mar 10 2020 at 01:49):
Pulling this into a separate topic (was in ValueSet Rendering):
Rob Hausam: And another issue with the rendering is that if (as in the above example) a compose.include.version is set, that is not displayed anywhere, and the text for "Expansion based on ..." may be incorrect. In this instance the SNOMED version has been set to the 20190731 version of the International Release (as that is what the current GPS is based on), but the text for the expansion says "Expansion based on SNOMED CT International edition 31-Jan 2020".
Lloyd McKenzie: I'd suggest creating a different issue for the release issue as I think that would be a different issue to fix.
Rob Hausam: Here's the tracker for the fixed version issue: FHIR#26462
Grahame Grieve:
the text for "Expansion based on ..." may be incorrectOn what grounds do you think it's incorrect?
Rob Hausam: @Grahame Grieve The reason I think it's incorrect is because (for example) in the value set definition compose.include.version is being set to http://snomed.info/sct/900000000000207008/version/20190731, but when the value set is expanded the message says:
Expansion based on SNOMED CT International edition 31-Jan 2020
So either the message is wrong, or the tx server isn't paying attention to the version specified in the definition (I'll verify which it is).
Grahame Grieve:
I'll verify which it isyes please
So it turns out that for $expand tx.fhir.org is ignoring the code system version, either specified as the 'system-version' parameter or in ValueSet.compose.include.version. And the server is also ignoring the 'activeOnly' parameter and ValueSet.compose.inactive. In all of the cases the expansions are using the latest version and include inactive concepts.
Grahame Grieve (Mar 10 2020 at 02:16):
do you have a parameters resource for the post submission that shows this?
Rob Hausam (Mar 10 2020 at 03:02):
yes, I have some that I'll post
Rob Hausam (Mar 10 2020 at 05:35):
Here are the Parameters resources:
346755004 |Treemix allergy maintenance kit (product)| is active in 20190731 but inactive in 20200131
expand_parameters-9.xml
expand_parameters-10.xml
expand_parameters-11.xml
expand_parameters-12.xml
407933006 |Varicella-zoster live attenuated vaccine injection (pdr for recon)+solvent (product)| and 419771007 |Measles/mumps/rubella/varicella vaccine powder and solvent for injection solution vial (product)| are active and subtypes of 108729007 |Varicella virus vaccine (product)| in 20190731 but are inactive and removed from the hierarchy in 20200131
expand_parameters-13.xml
expand_parameters-14.xml
expand_parameters-15.xml
Rob Hausam (Mar 10 2020 at 05:39):
I included status = 'active' in all of the ValueSet resources, as that is required, although the terminology server doesn't seem to care if it's there or not (we probably should fix that and generate it in the publishers?).
Grahame Grieve (Mar 11 2020 at 05:27):
we probably should fix that and generate it in the publishers?
Don't follow this bit?
Grahame Grieve (Mar 11 2020 at 05:27):
I've fixed the server - I think they all work now...
Rob Hausam (Mar 11 2020 at 11:43):
I'm not seeing a change in the behavior on tx.fhir.org - maybe it hasn't been updated/restarted yet?
Rob Hausam (Mar 11 2020 at 11:46):
The bit about "we probably should fix that and generate it in the publishers?" was suggesting that the ValueSet resource instance generated by the publisher for the valueSet parameter should include ValueSet.status (which it doesn't currently), as without that it's not a valid ValueSet resource instance.
Grahame Grieve (Mar 11 2020 at 12:56):
if the status is missing, you should get a validation error
Grahame Grieve (Mar 11 2020 at 12:56):
I haven't upgraded the server yet - sometime tomorrow
Rob Hausam (Mar 11 2020 at 14:41):
Here's a snippet from qa-tx.html - there is no ValueSet.status element (and no evidence of a validation error that I see here):
POST http://tx.fhir.org/r4/ValueSet/$expand?_limit=1000&_incomplete=true HTTP/1.0 User-Agent: Java FHIR Client for FHIR Accept: application/fhir+xml Content-Type: application/fhir+xml;charset=UTF-8 Accept-Charset: UTF-8 <?xml version="1.0" encoding="UTF-8"?><Parameters xmlns="http://hl7.org/fhir"><parameter><name value="profile-url"/><valueString value="dc8fd4bc-091a-424a-8a3b-6198ef146891"/></parameter><parameter><name value="system-version"/><valueString value="http://snomed.info/sct|http://snomed.info/sct/900000000000207008"/></parameter><parameter><name value="activeOnly"/><valueBoolean value="false"/></parameter><parameter><name value="includeDefinition"/><valueBoolean value="false"/></parameter><parameter><name value="excludeNested"/><valueBoolean value="true"/></parameter><parameter><name value="valueSet"/><resource><ValueSet><compose><include><system value="http://snomed.info/sct"/><version value="http://snomed.info/sct/900000000000207008/version/20190731"/><concept><code value="1985008"/><display value="Vomitus (substance)"/></concept><concept><code value="4386001"/><display value="Bronchospasm (finding)"/></concept><concept><code value="9826008"/><display value="Conjunctivitis (disorder)"/></concept><concept><code value="23924001"/><display value="Tight chest (finding)"/></concept><concept><code value="24079001"/><display value="Atopic dermatitis (disorder)"/></concept><concept><code value="31996006"/><display value="Vasculitis (disorder)"/></concept><concept><code value="39579001"/><display value="Anaphylaxis (disorder)"/></concept><concept><code value="41291007"/><display value="Angioedema (disorder)"/></concept><concept><code value="43116000"/><display value="Eczema (disorder)"/></concept><concept><code value="49727002"/><display value="Cough (finding)"/></concept><concept><code value="51599000"/><display value="Edema of larynx (disorder)"/></concept><concept><code value="62315008"/><display value="Diarrhea (finding)"/></concept><concept><code value="70076002"/><display value="Rhinitis (disorder)"/></concept><concept><code value="73442001"/><display value="Stevens-Johnson syndrome (disorder)"/></concept><concept><code value="76067001"/><display value="Sneezing (finding)"/></concept><concept><code value="91175000"/><display value="Seizure (finding)"/></concept><concept><code value="126485001"/><display value="Urticaria (disorder)"/></concept><concept><code value="162290004"/><display value="Dry eyes (finding)"/></concept><concept><code value="195967001"/><display value="Asthma (disorder)"/></concept><concept><code value="247472004"/><display value="Weal (disorder)"/></concept><concept><code value="267036007"/><display value="Dyspnea (finding)"/></concept><concept><code value="271757001"/><display value="Papular eruption (disorder)"/></concept><concept><code value="271759003"/><display value="Bullous eruption (disorder)"/></concept><concept><code value="271807003"/><display value="Eruption of skin (disorder)"/></concept><concept><code value="359610006"/><display value="Ocular hyperemia (disorder)"/></concept><concept><code value="410430005"/><display value="Cardiorespiratory arrest (disorder)"/></concept><concept><code value="418363000"/><display value="Itching of skin (finding)"/></concept><concept><code value="422587007"/><display value="Nausea (finding)"/></concept><concept><code value="698247007"/><display value="Cardiac arrhythmia (disorder)"/></concept><concept><code value="702809001"/><display value="Drug reaction with eosinophilia and systemic symptoms (disorder)"/></concept><concept><code value="768962006"/><display value="Lyell syndrome (disorder)"/></concept></include></compose></ValueSet></resource></parameter><parameter><name value="_limit"/><valueString value="1000"/></parameter><parameter><name value="_incomplete"/><valueString value="true"/></parameter></Parameters> HTTP/1.1 200 OK Connection: close Content-Type: application/fhir+xml; charset=UTF-8 Content-Length: 4498 Cache-control: public, max-age=600 Date: Sat, 07 Mar 2020 04:28:00 GMT Pragma: no-cache X-Request-Id: 268-185086 Access-Control-Allow-Origin: * Access-Control-Expose-Headers: Content-Location, Location Access-Control-Allow-Methods: GET, POST, PUT, PATCH, DELETE X-GDPR-Disclosure: All access to this server is logged as AuditEvent Resources, and these store your ip address (and logged in user, if one exists). Also, your IP address is logged with Google Analytics for building geomaps of server usage. Your continued use of the API constitutes agreement to these terms. See [link] for erasure requests Last-Modified: Sat, 07 Mar 2020 04:28:00 GMT Server: Health Intersections FHIR Server <?xml version="1.0" encoding="UTF-8"?><ValueSet xmlns="http://hl7.org/fhir"><expansion><identifier value="urn:uuid:2011c601-c5cb-4dce-98b4-9992f9662ea2"/><timestamp value="2020-03-07T04:28:00.111Z"/><parameter><name value="limitedExpansion"/><valueString value="-1"/></parameter><parameter><name value="excludeNested"/><valueString value="-1"/></parameter><parameter><name value="version"/><valueString value="http://snomed.info/sct|http://snomed.info/sct/900000000000207008/version/20200131"/></parameter><contains><system value="http://snomed.info/sct"/><code value="1985008"/><display value="Vomitus (substance)"/></contains><contains><system value="http://snomed.info/sct"/><code value="4386001"/><display value="Bronchospasm (finding)"/></contains><contains><system value="http://snomed.info/sct"/><code value="9826008"/><display value="Conjunctivitis (disorder)"/></contains><contains><system value="http://snomed.info/sct"/><code value="23924001"/><display value="Tight chest (finding)"/></contains><contains><system value="http://snomed.info/sct"/><code value="24079001"/><display value="Atopic dermatitis (disorder)"/></contains><contains><system value="http://snomed.info/sct"/><code value="31996006"/><display value="Vasculitis (disorder)"/></contains><contains><system value="http://snomed.info/sct"/><code value="39579001"/><display value="Anaphylaxis (disorder)"/></contains><contains><system value="http://snomed.info/sct"/><code value="41291007"/><display value="Angioedema (disorder)"/></contains><contains><system value="http://snomed.info/sct"/><code value="43116000"/><display value="Eczema (disorder)"/></contains><contains><system value="http://snomed.info/sct"/><code value="49727002"/><display value="Cough (finding)"/></contains><contains><system value="http://snomed.info/sct"/><code value="51599000"/><display value="Edema of larynx (disorder)"/></contains><contains><system value="http://snomed.info/sct"/><code value="62315008"/><display value="Diarrhea (finding)"/></contains><contains><system value="http://snomed.info/sct"/><code value="70076002"/><display value="Rhinitis (disorder)"/></contains><contains><system value="http://snomed.info/sct"/><code value="73442001"/><display value="Stevens-Johnson syndrome (disorder)"/></contains><contains><system value="http://snomed.info/sct"/><code value="76067001"/><display value="Sneezing (finding)"/></contains><contains><system value="http://snomed.info/sct"/><code value="91175000"/><display value="Seizure (finding)"/></contains><contains><system value="http://snomed.info/sct"/><code value="126485001"/><display value="Urticaria (disorder)"/></contains><contains><system value="http://snomed.info/sct"/><code value="162290004"/><display value="Dry eyes (finding)"/></contains><contains><system value="http://snomed.info/sct"/><code value="195967001"/><display value="Asthma (disorder)"/></contains><contains><system value="http://snomed.info/sct"/><code value="247472004"/><display value="Weal (disorder)"/></contains><contains><system value="http://snomed.info/sct"/><code value="267036007"/><display value="Dyspnea (finding)"/></contains><contains><system value="http://snomed.info/sct"/><code value="271757001"/><display value="Papular eruption (disorder)"/></contains><contains><system value="http://snomed.info/sct"/><code value="271759003"/><display value="Bullous eruption (disorder)"/></contains><contains><system value="http://snomed.info/sct"/><code value="271807003"/><display value="Eruption of skin (disorder)"/></contains><contains><system value="http://snomed.info/sct"/><code value="359610006"/><display value="Ocular hyperemia (disorder)"/></contains><contains><system value="http://snomed.info/sct"/><code value="410430005"/><display value="Cardiorespiratory arrest (disorder)"/></contains><contains><system value="http://snomed.info/sct"/><code value="418363000"/><display value="Itching of skin (finding)"/></contains><contains><system value="http://snomed.info/sct"/><code value="422587007"/><display value="Nausea (finding)"/></contains><contains><system value="http://snomed.info/sct"/><code value="698247007"/><display value="Cardiac arrhythmia (disorder)"/></contains><contains><system value="http://snomed.info/sct"/><code value="702809001"/><display value="Drug reaction with eosinophilia and systemic symptoms (disorder)"/></contains><contains><system value="http://snomed.info/sct"/><code value="768962006"/><display value="Lyell syndrome (disorder)"/></contains></expansion></ValueSet>
Grahame Grieve (Mar 11 2020 at 19:36):
error should be qa.html
Rob Hausam (Mar 11 2020 at 20:37):
right - that means you have to coordinate between the two pages rather than having a directly corresponding reference
but in this case I don't see any related errors, warnings or info messages (including the suppressed ones) in the qa.html
plus, if I go back to a couple of the Parameters resource examples that I sent and remove the ValueSet.status line and I post to tx.fhir.org directly (in Postman), the terminology server still processes the request without generating any errors or warnings (or any indication of any kind that there is something invalid about the ValueSet resource)
Grahame Grieve (Mar 11 2020 at 20:39):
right. the terminology server doesn't need status to do an expansion, so it doesn't check for it.
Grahame Grieve (Mar 11 2020 at 20:39):
what IG are you net getting a warning?
Rob Hausam (Mar 11 2020 at 20:43):
yes, that makes sense from an operational perspective - if that's just an internal shortcut that the publisher uses then I can understand not flagging that as a qa issue - but because it's not completely internal (you can see it in qa-tx.html) and because it would be trivial to add for completeness, the question is shouldn't we go ahead and do that so that the ValueSet resource instances that are submitted within the parameters would actually be conformant?
Rob Hausam (Mar 11 2020 at 20:43):
the IG is IPS
Grahame Grieve (Mar 11 2020 at 20:44):
which value set?
Rob Hausam (Mar 11 2020 at 20:48):
it's what the publisher submits to tx.fhir.org for any (and all) of the ValueSet resource instances in the IG (as far as I've seen)
all of the ValueSet resources instances in the IG have ValueSet.status and are conformant, but when they are submitted by the publisher as a ValueSet instance in the $expand parameters only the 'compose' element is being sent
Rob Hausam (Mar 11 2020 at 20:51):
as you said (and I agreed) that works - but since it isn't conformant to the spec, is that the way we should do it?
Grahame Grieve (Mar 11 2020 at 21:02):
ok I think I found the one place that I missed setting status
Grahame Grieve (Mar 11 2020 at 22:48):
ok upgraded the server
Rob Hausam (Mar 11 2020 at 22:50):
I noticed it was down momentarily so figured that was the case
Rob Hausam (Mar 11 2020 at 22:50):
I'll check it out
Rob Hausam (Mar 11 2020 at 22:51):
I already looked at the commits
Rob Hausam (Mar 12 2020 at 05:29):
So what I'm seeing is that tx.fhir.org is now responding correctly to the activeOnly parameter, but it is still ignoring ValueSet.compose.inactive.
Rob Hausam (Mar 12 2020 at 05:43):
Also, irrespective of specifying a particular SNOMED CT version (in addition to the edition) in system-version or ValueSet.compose.include.version, the expansion is done on the current version. I think that's actually expected, since the SNOMED import for the server is done on the snapshot release, which only includes the content for that specific version of the chosen edition. In order to support multiple versions for an edition it would be necessary to import the full release and handle the content (including the transitive closure) for any specified version. That's doable, but would certainly require additional work to implement. An alternative could be to allow value set instances defined for fixed versions of the code system(s) to include their own expansions (which the tx server would simply pass through and wouldn't need to calculate). Is that something to consider for the tx server (we definitely would like to have it in IPS for the GPS value sets)? @Grahame Grieve
Rob Hausam (Mar 12 2020 at 05:54):
Based on the comment in terminology here I see that my comment above isn't accurate. I'm assuming that the way we can now (as of this release) manage multiple versions of the same SNOMED CT edition is to generate the cache and host multiple specified versions (not any arbitrary version as with the full release)? I'm assuming that's a good guess (and you can correct it if it isn't). It's definitely good to have the option.
Grahame Grieve (Mar 12 2020 at 05:56):
ok I agree that I ignore ValueSet.compose.inactive (when did that get added?), but I'm pretty sure that the expansion is done on the version you ask for
Grahame Grieve (Mar 12 2020 at 05:57):
and you shouldn't need to do anything fancy to have value sets based on different versions in the same ig
Rob Hausam (Mar 12 2020 at 06:03):
yeah, that sounds good - we just don't have the other versions loaded at the moment
I'll work on that
ValueSet.compose.inactive was in STU3 - added in 1.8.0 :)
Michael Lawley (Mar 12 2020 at 06:10):
Is there a FHIR CodeSystem that corresponds to the valid values for the http://hl7.org/fhir/concept-properties#status property?
Grahame Grieve (Mar 12 2020 at 06:19):
I think there is in the current build
Grahame Grieve (Mar 12 2020 at 06:19):
if not, it's because I haven't merged it in yet
Last updated: Apr 12 2022 at 19:14 UTC
