Stream: terminology / utg
Topic: Content in Publisher Infrastructure
Ted Klein (Oct 16 2021 at 19:45):
@Grahame Grieve In the discussions with @John Moehrke around the IHE FormatCodes content, he asserted that the reason the value set for FormatCodes in IHE renders in the FHIR R4 spec is "you did some ftp magic" whatever that means. @Lisa Nelson has put in a UTG request to add the value set for format codes into THO as a CDA ValueSet, and @Marc Duteau assisted her with the resource definition. He submitted it and I have been trying to figure out why it will not build and expand the value set no matter what I do - the identical <compose> and parameters as the FHIR R4 value set definition will not expand in a UTG/THO build. I can only conjecture that this was in the infrastructure only temporarily, or only accessed in the FHIR R4 build. The value set has been removed for FHIR R5 apparently, and it was one of those that was supposed to have been 'moved' to THO. I have verified with John that the underlying code system with the correct (as per IHE) <url> exists and is resolvable to the resource (http://ihe.net/fhir/ihe.formatcode.fhir/CodeSystem/formatcode) and John said that the IHE IG expands against it just fine in the Publisher. In trying to get to the bottom of all this, it seems the canonical URL http://terminology.hl7.org/CodeSystem/hl7-document-format-codes does NOT exist in THO, but DOES exist in the publishing infrastructure somewhere as well; a compose like: <compose>
<include>
<system
value="http://terminology.hl7.org/CodeSystem/hl7-document-format-codes"/>
</include>
</compose>
Ted Klein (Oct 16 2021 at 19:47):
happily expands to all the codes without error. The content cannot be found in FHIR or in THO. Where is it coming from? And why since the URL was obviously changed to THO from hl7.org/fhir, was the content not moved but it was left in the infrastructure somewhere?
Ted Klein (Oct 16 2021 at 19:47):
@Grahame Grieve that is the SDWG bit. The IHE bit - as I understand it we (HL7) have three choices:
Ted Klein (Oct 16 2021 at 19:48):
- Create a 'convenience copy' of the IHE code system in THO (and take on the burden of maintenance and the MOU to do so)
Ted Klein (Oct 16 2021 at 19:49):
- Access it directly from IHE with a depends-on parameter to the IHE IG that contains the code system source of truth (thus taking on the risk that if IHE moves it or does something with it that implies a ValueSet CLD change it breaks the THO copy and valuesets built on it)
Ted Klein (Oct 16 2021 at 19:50):
- Download a 'secret copy' of the content and stuff it in tx.fhir.org or somewhere else in the infrastructure (taking on the burden of #1 above but without the MOU)
Ted Klein (Oct 16 2021 at 19:56):
Any other options I missed? @John Moehrke has told me he started the process of requesting an HTA page for the IHE format codes - is that accurate @Carol Macumber ? I would really like to figure this out for @Lisa Nelson and the CDA gang for the THO release coming up, but of course the combined value set CLD which includes the IHE specific content in one <include> and the HL7 content in the other <include> fails to do any expanding, as the expansion operation fails when it is unable to access the IHE content at http://ihe.net/fhir/ihe.formatcode.fhir/CodeSystem/formatcode. Please also note the the URL used for this content in FHIR R4 is http://ihe.net/fhir/ValueSet/IHE.FormatCode.codesystem which also fails in the Publisher as it is an old and invalid URL (as asserted by John) and you must have dispensed with it after the R4 publication.
Ted Klein (Oct 16 2021 at 19:57):
@Grahame Grieve please help me understand and untangle this mess. I am happy to help Lisa, John, Marc, and Carol to get all the pieces aligned, but I am ignorant of where the content we can see is coming from in the Publisher, and am seeking your opinion on best course of action to be able to browsable include the IHE content in THO (it would be super nice to use the actual real URL which resolved to the page of the rendered resource)
Rob Hausam (Oct 18 2021 at 02:49):
A CodeSystem resource for this code system with the http://terminology.hl7.org/CodeSystem/hl7-document-format-codes canonical url does exist in the packages repository in 'fhir.tx.support.r4', and therefore that code system content (8 codes - 2 each for C-CDA 1.1, C-CDA 2.1, C-CDA 2.2 and PACP 1.0) is available on tx.fhir.org. The commit that added that CodeSystem resource instance was made by Grahame on Mar 3, 2020.
Ted Klein (Oct 18 2021 at 16:59):
@Rob Hausam I assumed so, as the value set built on that 'all codes' expands fine. The issue is the with IHE codes, which are in the R4 expansion but I cannot find any way to access that content in THO currently. This one that you reference is only the SDWG CDA additions to the IHE code system.
Rob Hausam (Oct 18 2021 at 17:16):
Yes, correct.
Ted Klein (Oct 18 2021 at 19:08):
Any clue where the IHE codes that are in the expansion in R4 came from? They do not appear to be in the infrastructure anywhere I can find.
Ted Klein (Oct 19 2021 at 12:23):
These are the ones I am talking about: https://profiles.ihe.net/fhir/ihe.formatcode.fhir/CodeSystem-formatcode.html
Ted Klein (Oct 19 2021 at 12:23):
FHIR R4 expansion: http://hl7.org/fhir/valueset-formatcodes.html
Rob Hausam (Oct 19 2021 at 20:30):
It seems like what we need and want to do is add an additional include for the HL7 code system to the formatcodes value set definition. I believe that's what @John Moehrke has been wanting? @Ted Klein
Ted Klein (Oct 19 2021 at 20:51):
UP-241. this is what I have been trying to get working. I commented out the IHE code system compose.include as it kills the expansion completely. If you read carefully above, what I verified experimentally is that the IHE content which was in the system somewhere when R4 was a published as it is in the R4 value set expansion, and referred to by what John claimed was an erroneous and obsolete code system <url> has somehow been removed and none of the half dozen possible URIs that I tried will surface the IHE content - including the URI used in the R4 value set, which will not expand on a build; it appears to no longer be anywhere in the infrastructure. THO will have the value set including the HL7 content when UP-241 is approved and implemented, and Lisa stated unequivocally to me earlier today that she wants the HL7 code system to be an explicit THO resource, browsable through the THO interface. You could probably make that easier by pulling the xml resource of the HL7 code system with the CDA content out of the package and giving it to me and Marc Duteau to be added to UP-241.
John Moehrke (Oct 19 2021 at 21:03):
the valueSet that should live in FHIR should pull in both IHE codeSystem, and the HL7 codeSystem.
But, as Ted explains, this doesn't work for the reasons Ted explains. The HL7 codeSystem does not seem to be in THO, and the IHE codeSystem needs to somehow be made available beyond the IG that IHE has published.
John Moehrke (Oct 19 2021 at 21:04):
IHE has a valueSet much like this in their FormatCode IG. This however is IHE managed ValueSet. The reason for a FHIR managed ValueSet is for the purposes of FHIR defined DocumentReference resource.
John Moehrke (Oct 19 2021 at 21:04):
IHE publication -- https://profiles.ihe.net/fhir/ihe.formatcode.fhir/index.html
John Moehrke (Oct 19 2021 at 21:05):
IHE codeSystem canonical URI -- http://ihe.net/fhir/ihe.formatcode.fhir/CodeSystem/formatcode
John Moehrke (Oct 19 2021 at 21:05):
IHE 'similar' valueSet canonical URI -- http://ihe.net/fhir/ihe.formatcode.fhir/ValueSet/formatcode
John Moehrke (Oct 19 2021 at 21:07):
the IHE 'similar' valueSet does find the HL7 codes, as it has in it an "compose.include"
{
"system" : "http://terminology.hl7.org/CodeSystem/hl7-document-format-codes"
},
John Moehrke (Oct 19 2021 at 21:08):
the IG publisher does find and pull those codes into the IHE valueSet expansion.
John Moehrke (Oct 19 2021 at 21:08):
@Ted Klein I don't know what tooling you are using that does not find the hl7 codes.
John Moehrke (Oct 19 2021 at 21:08):
but I also don't know the magjic that the IG publisher is using when the IHE formatcode IG does find them
John Moehrke (Oct 19 2021 at 21:09):
@Grahame Grieve magic
Ted Klein (Oct 19 2021 at 21:10):
@John Moehrke I went over the options with @Lisa Nelson earlier today. If I understood correctly, she prefers to have a "convenience copy" of the IHE code system in THO since the IG Publisher and various servers cannot easily directly reference the source of truth in IHE beyond the IG that it is defined in. This was done for SNODENT for basically the same reasons, and for CMS Present On Admission codes system. But it is an HTA decision and agreement with you for such a thing.
John Moehrke (Oct 19 2021 at 21:11):
yes. I think I agree. The mechanism for 'convenience copy' is not known to me, but if it can be automatic following IHE versioning of the IG that would be best for me. If it can't be automatic, then I need to know how to kick, the system when IHE versions their IG. I don't think this versioning of the IHE IG will happen often, usually once a year.
Ted Klein (Oct 19 2021 at 21:21):
@John Moehrke you are 100% on the money here. The things to make it happen are:
Ted Klein (Oct 19 2021 at 21:21):
- Agreement by HTA with concurrence from TSMG
Ted Klein (Oct 19 2021 at 21:22):
- Agreement worked out by HTA between HL7 and IHE for HL7 to publish this copy of IHE content (copyright, IP issues, etc)
John Moehrke (Oct 19 2021 at 21:22):
I agree
Ted Klein (Oct 19 2021 at 21:22):
- Process to keep it current
John Moehrke (Oct 19 2021 at 21:22):
still agree
Ted Klein (Oct 19 2021 at 21:24):
Item 3 is mostly manual as we have no tooling or automagical gizmos to "know" when you update your version. My suggestion is that when you update it, you send a message to HL7 (which person exactly is not yet clear) that a it has been updated, and ideally a copy of the new content that can just be 'slapped in' to make the update happen.
Ted Klein (Oct 19 2021 at 21:24):
We have TWO options for such a 'convenience copy':
Ted Klein (Oct 19 2021 at 21:25):
A. We have an xml resource marked as external code system with complete content (see the CMS Present on Admission example), but with the description having the URL to your master source of truth for browsing and perhaps download;
Ted Klein (Oct 19 2021 at 21:27):
B. We have a 'hidden' copy in the 'infrastructure' which is probably a load of your content into thee package set that is the tx.fhir.org content. The THO code system resource exists also but is flagged as metadata only with no content, and points to your page at IHE.
Ted Klein (Oct 19 2021 at 21:27):
Either one permits folks to see the content and to build IGs with value sets that include it with no errors
Ted Klein (Oct 19 2021 at 21:27):
solution A has the issue of a duplicate copy and need for timely update
Ted Klein (Oct 19 2021 at 21:29):
B has the issue that it works for THO and the IG Publisher, but not so well for other servers that want to have the content for validation purposes, or folks that use their own server for IG Publisher operations (you can override it in the IG control file and use a different one than tx.fhir.org).
Ted Klein (Oct 19 2021 at 21:30):
example of B is the metadata code system resources in THO for UCUM, LOINC, and SNOMED.
John Moehrke (Oct 19 2021 at 21:39):
Since IHE (myself) used the IG publisher, and IHE is hooked into the ig-registry.... there is a way HL7 can know when a version happens.
John Moehrke (Oct 19 2021 at 21:41):
- but I fully recognize "can know" is not all that helpful without the "mechanism is deployed that does do something"
Ted Klein (Oct 19 2021 at 21:46):
you are reading my mind, my good man...
Ted Klein (Oct 19 2021 at 21:47):
I would let Lisa and Carol M know so that they can decide how they want to move forward. My sense is the technical bits are the easiest part; figuring out the organizational stuff and getting some decisions will likely take the longest.
Rob Hausam (Oct 20 2021 at 00:51):
Yes, agree.
Grahame Grieve (Oct 20 2021 at 03:42):
trying to keep track of all this is super confusing.
Grahame Grieve (Oct 20 2021 at 03:42):
here's the source of the definitions:
Grahame Grieve (Oct 20 2021 at 03:42):
Grahame Grieve (Oct 20 2021 at 03:42):
I added it back in Mar 2020 but I have no memory of why. Presumably it didn't have a home
Grahame Grieve (Oct 20 2021 at 03:44):
I have an email record which is where I got it from that says it was to be added to UTG but obviously that never happened, so I added to our backup instead, I guess
Grahame Grieve (Oct 20 2021 at 03:45):
so:
- hl7 defines some codes in a code system that should live in THO, but doesn't. is the list wrong? incomplete?
- ihe defines some codes in a code system that has problems? or no? I can't tell
- there's an actual problem here somewhere?
Grahame Grieve (Oct 20 2021 at 03:45):
the conversation is pretty confusing....
Rob Hausam (Oct 20 2021 at 03:48):
Well, the conversation been going on in various bits at a time for over 2 years now - so it's easy to get lost in the details of where it is and where it was supposed to be getting to.
Grahame Grieve (Oct 20 2021 at 03:48):
right. so can we get some clarity? what's our functional problems, and where do we want to get to?
Grahame Grieve (Oct 20 2021 at 03:52):
anyway, for future reference: CodeSystem definitions can come from one of 4 places:
- innate to tx.fhir.org - big external code systems, and code systems with grammars
- the base FHIR specification
- fhir.tx.support
- THO
- any content defined in IGs, or in referenced IGs.
There's nowhere else for it come from
Rob Hausam (Oct 20 2021 at 04:05):
Apparently we don't have an HL7 value set yet that is defined to contain both of the sets of HL7 and IHE defined format codes and can be successfully expanded in the FHIR build process. I think that is the most important thing that is missing, and then there are also the questions of source of truth and visibility and accessibility (as well as maintainability going forward). I'll let @John Moehrke and @Ted Klein add which ones of those they think are needed and where they should be located.
Grahame Grieve (Oct 20 2021 at 04:07):
it would be unusual to have an HL7 value set that contains bot the HL7 and IHE defined codes. Particularly if it's referenced in THO / FHIR c ore. That would really imply the IHE definitions are contained in THO, since FHIR core depends on THO and THO depends on FHIR core - not adding another external package to that deadly embrace
Grahame Grieve (Oct 20 2021 at 04:07):
does HL7 really need the IHE defined codes?
John Moehrke (Oct 20 2021 at 11:21):
There is a ValueSet that has been authored, but it will not "build". I am not sure what tool Ted is trying to use to build it.
John Moehrke (Oct 20 2021 at 11:22):
The FHIR DocumentReference needs a ValueSet, that will be example going forward, that pulls some codes from IHE and some codes from HL7. We just want that to succeed.
John Moehrke (Oct 20 2021 at 11:24):
I have noted that the IHE IG that defines their CodeSystem does also have a ValueSet that pulls in all of the IHE codes, AND also the HL7 codes. It succeeds to build using the IG builder. (again, I am not sure what tool Ted is using that is failing)
Grahame Grieve (Oct 20 2021 at 16:57):
there isn't a path to having IHE codes in the base spec unless they are defined in the base spec or THO
Grahame Grieve (Oct 20 2021 at 16:58):
dicom might be different
John Moehrke (Oct 20 2021 at 20:43):
I thought that there was interest in something called "a convenience copy"? is that the same thing as is done for LOINC, SNOMED, DICOM... right?
Grahame Grieve (Oct 20 2021 at 21:57):
that's what we have now, and I got the impression that it's a problem
John Moehrke (Oct 20 2021 at 22:02):
if that is what we got, then I think I am happy. Not sure then what is left broken.
Ted Klein (Oct 21 2021 at 17:48):
ok many questions sorry was offline in airplanes all day yesterday just catching up now today...here is the deal:
Ted Klein (Oct 21 2021 at 17:53):
FHIR R4 has a value set http://hl7.org/fhir/valueset-formatcodes.html that contains the IHE content. It was NOT on the docket to be migrated to THO as it is 'external' content (IHE content). A new UTG proposal for a value set to contain BOTH the IHE content and the CDA content for extended format codes was put in, and I verified with Lisa that indeed this is the desire. In attempting to validate it when submitted, it failed to build with errors in the <compose> that included the IHE content (using the same <url> value as the value set in R4 above). I messed with it A LOT, and verified that the <compose> with the CDA content DOES expand if the IHE content composes are commented out. This compose is:
Ted Klein (Oct 21 2021 at 17:53):
<include>
<system value="http://terminology.hl7.org/CodeSystem/hl7-document-format-codes"/>
</include>
Ted Klein (Oct 21 2021 at 17:54):
This code system may have been scheduled to be put into THO through UTG, but the ball must have been dropped somewhere some time ago and it is NOT in THO - however, Rob Hausam verified that it IS in tx.fhir.org; unknown when or who loaded it (unimportant).
Ted Klein (Oct 21 2021 at 17:56):
The IHE code system is http://ihe.net/fhir/ihe.formatcode.fhir/CodeSystem/formatcode and it is the only content (my understanding; may be erroneous) in the IHE IG at https://profiles.ihe.net/fhir/ihe.formatcode.fhir/index.html
Ted Klein (Oct 21 2021 at 17:56):
If this <url> is in the compose in a THO value set, it does NOT expand
Ted Klein (Oct 21 2021 at 17:57):
So...in my understanding, and please correct me if I am out in the zones here @Grahame Grieve what needs to be done, working from the end result backwards is:
Ted Klein (Oct 21 2021 at 17:58):
- get the HL7 format codes into THO where they are browsable and maintained through the UTG process, and can be used in any value set anywhere being built referencing them, and are part of the THO download package once the release is done;
Ted Klein (Oct 21 2021 at 17:59):
- Get the IHE code system with the correct system uri in the compose to be processed by the Publisher so the combined value set will expand;
Ted Klein (Oct 21 2021 at 17:59):
- Get the combined value set through UTG and into the THO ci build and the Release.
Ted Klein (Oct 21 2021 at 18:00):
For #2 above, it is my understanding (again, correct me if I am ignorant and confused here @Grahame Grieve ) that this can be done one of three possible ways, all of which have pros and cons, but some of which are much more tractable:
Ted Klein (Oct 21 2021 at 18:02):
2a. Have the THO build include a 'depends-on' for the IHE IG containing the format codes. Pros: easy, no extra work. Cons: makes THO builds depending upon being able to access and include the IHE IG when a ci build or THO release is done. Unknown (to me) is whether or not the THO build will fail if the depends-on cannot be done, or it is a non-fatal error (for instance if IHE is doing server maintenance or updating the IG or whatever)
Ted Klein (Oct 21 2021 at 18:05):
2b. Include a 'convenience copy' of the IHE code system in THO. PROS: no dependencies for THO builds. CONS: no process defined or resources identified for updating the content if IHE updates it, thus it gets out of sync with IHE. Also, need an agreement to do so between HL7 and IHE, put together through the HTA process and approved by HL7 (may be time consuming and contentious);
Grahame Grieve (Oct 21 2021 at 18:05):
#2 is probably the go, but this conversation would be able to go somewhere if this was specific:
it failed to build with errors in the <compose> that included the IHE content
what was the error?
Ted Klein (Oct 21 2021 at 18:07):
2c. Have IHE make the code system available with some way for the Publisher to access it when needed for an expansion, some kind of API or endpoint or something. PROS: Publisher can directly access code systems that are stood up by other organizations to be able to be included in FHIR builds, the content is always up to date. CONS: huge development task for both the IG Publisher and the content owner.
Ted Klein (Oct 21 2021 at 18:08):
Grahame, just say you were online good we can continue...
Ted Klein (Oct 21 2021 at 18:08):
there is a 2d:
Ted Klein (Oct 21 2021 at 18:09):
a 'convenience copy' which is included int he infrastructure like we do with LOINC, SNOMED, country codes, others, either in a cache content file or in the tx.fhir.org package so it is "in there" but no one really knows about it.
Ted Klein (Oct 21 2021 at 18:10):
for all #2 we need a code system resource for formatcodes which is a 'stub' ie code system metadata record, which MUST identify where the source of truth of the content it, as well as where the HL7 convenience copy is and how it is to be updated when necessary.
Ted Klein (Oct 21 2021 at 18:11):
as to your question @Grahame Grieve I cannot find the damn FHIR page with all the parameters for the ig.ini file so I can test what it does and what the error is, which I desperately want to do (and can do easily I think). I simply do not know the syntax for the parameter for the depeends-on so I can test it and tell you what the error is
Grahame Grieve (Oct 21 2021 at 18:12):
you don't need any depends on. Just try and build with the compose as Lisa proposed, and tell us what the error is
Ted Klein (Oct 21 2021 at 18:12):
if the publisher merely gives an error that the IG cannot be included and another error that the code system in the value set <compose> is not accessible, then #1 may be the easiest by far.
Ted Klein (Oct 21 2021 at 18:13):
but if it causes the whole build to fail, then it is unacceptable IMHO.
Ted Klein (Oct 21 2021 at 18:13):
I will do that now, give me the 25m or so for the build to run...
Ted Klein (Oct 21 2021 at 18:14):
but when I tried it last week, the value set expansion cannot be build and contains zero concepts becuyse the codes system cannot be found. Here is the compose:
Ted Klein (Oct 21 2021 at 18:16):
<compose>
<!-- From the IHE value set -->
<include>
<system value="http://ihe.net/fhir/ihe.formatcode.fhir/CodeSystem/formatcode"/>
<filter>
<property value="notSelectable"/>
<op value="="/>
<value value="false"/>
</filter>
</include>
<!-- From what is in Grahame's R4 for the code system
<include>
<system value="http://ihe.net/fhir/ValueSet/IHE.FormatCode.codesystem"/>
</include>
-->
<include>
<system value="http://terminology.hl7.org/CodeSystem/hl7-document-format-codes"/>
</include>
<!--
<include>
<system value="http://dicom.nema.org/resources/ontology/DCM"/>
<concept>
<code value="1.2.840.10008.5.1.4.1.1.88.59"/>
<display value="Key Object Selection Document"/>
</concept>
</include>
-->
</compose>
Ted Klein (Oct 21 2021 at 18:16):
The DICOM include I commented out as it also breaks the expand of the value set.
Ted Klein (Oct 21 2021 at 18:17):
what is it supposed to do?
Grahame Grieve (Oct 21 2021 at 18:25):
include the code. I would expect both of those includes to work as is
Grahame Grieve (Oct 21 2021 at 18:25):
so what's the error?
Last updated: Apr 12 2022 at 19:14 UTC
