Stream: fmg
Topic: IGs that try to overrule the core spec
Lloyd McKenzie (Jun 26 2021 at 02:29):
I learned today that the most recent US Core publication (which we recently approved) attempts to loosen the rules on Extensible bindings (relevant language is here). I've got a serious problem with that. Methodologically, it's not allowed. And for it to even be attempted is a serious process issue.
The problem is that if there's an extensible binding to a value set that includes an overarching SNOMED (or other) concept that completely encompasses the space, then there can never be a situation where one of the codes won't apply and thus a code from the value set must always be present. The issue is that the main EHR implementers object to sending something like a generic SNOMED code for 'procedure' when they're sending free text or a local code that they don't have an ability to translate.
The thing is that the current rules for 'Extensible' don't say "make your best effort" or "if you can do so automatically". The rule is "if a mapping is possible, the value set code must be used". It's not intended to be used at all in the way US Core is trying to use it. It's intended to be used in such situations as "all NDC codes" where if a drug happens to not have an NDC code, you're free to use just text or a local code.
I understand that the existing binding strengths don't really give the US core implementers an ability to say what they'd like to say (which, as I understand it, is essentially "for net new content you control, you SHALL use a fine-grained code from our value set, for legacy or external content, you SHOULD map to the value set. While doing so, totally ignore all of the high level SNOMED codes we inherit from our high-level intensional definition because it's too impractical to try to exclude them"). In practice, they could establish a 'preferred' binding and set those rules with text, but they don't want to do that because then tools wouldn't spit out warnings. (Technically, they could get around that with a warning invariant.)
In any event, we shouldn't be publishing IGs that try to say that the rules in the base spec don't apply - especially when it's being done by an IG that's as foundational as US Core. And I'm particularly concerned that even though the authors of the IG understood the rules, they didn't seek any vetting by MnM, Vocab or the FMG when they decided to write language that ignores those rules.
Josh Mandel (Jun 26 2021 at 14:23):
Two things, and both about the specifics of this case rather than a management perspective on whether it's okay...
-
Practically speaking, is this just an issue because the values in the target ValueSets include certain overarching terms that they shouldn't? I don't know how to actually review this value set, because I can't find a FHIR representation of it and I just see a bunch of blank template type stuff on the the vsac page. (Why do we allow IGs to reference value sets in this way??)
-
"Also for CodeableConcept if only text is available, then just text may be used." -- irrespective of the ValueSet we're binding to, is this language consistent with FHIR's definition of extensible?
Lloyd McKenzie (Jun 26 2021 at 21:06):
-
That's one of the problems, but not the only one and realistically not the main one. In most cases, whatever local code or text is present from an external or legacy source will have a translation to a fine-grained code (SNOMED or otherwise). Even if all the high-level codes were removed, an extensible binding says that if a code from the value set applies, it must be present. That would mean that all legacy and external codes would need to be mapped. The real issue is that US Core wants 2 sets of rules - one for new content authored by the conformant system and one for all other content. However, there's nothing in the instances that differentiates the two cases. Also, there's nothing in current regulation that deals with differentiating them.
-
Text vs. code doesn't matter. The specific wording for extensible is "To be conformant, the concept in this element SHALL be from the specified value set if any of the codes within the value set can apply to the concept being communicated." - it doesn't matter whether the concept being communicated by the CodeableConcept is in CodeableConcept.text, CodeableConcept.coding or CodeableConcept.extension. So the wording in the IG is absolutely not consistent with the SHALL rule in the core spec.
Grahame Grieve (Jun 26 2021 at 21:13):
I'm not sure what this sentence means:
For data not captured by the system transmitting the information, the coded data should be automatically converted to a fine-grained code from the specified value set.
Grahame Grieve (Jun 26 2021 at 21:15):
I note that the passage doesn't actually say what Lloyd says it says.
it says:
US Core guidance provides more flexibility for situations where implementers cannot fully comply with the FHIR base guidance
But the guts of the advice says:
If this is not possible, the system can provide the existing code or the free text, and a high-level abstract code, such as SNOMED CT ‘Procedure’, to remain conformant with the extensible binding
This appears to me to meet the technical requirements of extensible even if it does not meet the spirit of the rule
Grahame Grieve (Jun 26 2021 at 21:18):
I don't know how to actually review this value set, because I can't find a FHIR representation of it and I just see a bunch of blank template type stuff on the the vsac page. (Why do we allow IGs to reference value sets in this way??)
I'm not sure why you can't reference this value set - the vsac reference should work and take to a list of codes. it does for me
And we let IGs reference value sets this way because that's the right value set to reference, and we're still trying to move VSAC towards a better approach
Grahame Grieve (Jun 26 2021 at 21:19):
@Lloyd McKenzie are we sure they didn't talk to vocab?
Lloyd McKenzie (Jun 26 2021 at 21:29):
I didn't see any discussion about it on the list or Zulip. @Rob Hausam might know if this language was approved on a call (in which case, there's an issue with Vocabulary's actions as well...)
Providing a high-level abstract code would be fine. Unfortunately, the section also says
Also for CodeableConcept if only text is available, then just text may be used. - which isn't true for any of the elements where high-level SNOMED codes are in the value set and is only true for others if the text can't be mapped. (Not 'system isn't capable of mapping' but 'no human, taking enough time, can map it'.)
The US Core spec also says
Although the FHIR guidance for extensible bindings indicates that all conceptual overlaps including free text be mapped the coded values in the bindings, US Core guidance provides more flexibility for situations where implementers cannot fully comply with the FHIR base guidance. This flexibility is sometimes necessary and expected for legacy and text only data. For newly recorded, non legacy data, a system SHOULD meet the conformance of the value set.
It's non-conformant to soften a SHALL in the core spec to a SHOULD in an IG. (And in this case, they're saying that the SHOULD doesn't even apply to all cases.)
Grahame Grieve (Jun 26 2021 at 21:45):
so your issue is text only?
Lloyd McKenzie (Jun 26 2021 at 23:01):
My issue is both quotes. Text only breaks the rules. So does softening SHALL to SHOULD and saying that binding only applies to some types of instances that claim conformance with the profile, not all.
Grahame Grieve (Jun 26 2021 at 23:01):
saying that binding only applies to some types of instances that claim conformance with the profile
how did they do that?
Lloyd McKenzie (Jun 26 2021 at 23:05):
This text: "Although the FHIR guidance for extensible bindings indicates that all conceptual overlaps including free text be mapped the coded values in the bindings, US Core guidance provides more flexibility for situations where implementers cannot fully comply with the FHIR base guidance. This flexibility is sometimes necessary and expected for legacy and text only data. For newly recorded, non legacy data, a system SHOULD meet the conformance of the value set."
Lloyd McKenzie (Jun 26 2021 at 23:06):
That's not a type of flexibility they can add. And they can't make a SHALL a SHOULD.
Grahame Grieve (Jun 26 2021 at 23:25):
but then they said,
the system can provide the existing code or the free text, and a high-level abstract code, such as SNOMED CT ‘Procedure’, to remain conformant with the extensible binding
so I think you're jumping at a shadow on that one. That's legally compliant, even they imply it's not.
Lloyd McKenzie (Jun 27 2021 at 01:51):
At minimum, the language is unclear. With the number of SHOULDs and exclusions, it's certainly not clear that every single Procedure.code and AllergyIntolerance.code SHALL have a code from the value set present, no matter what.
Rob Hausam (Jun 27 2021 at 02:09):
I don't recall that anyone representing US Core has come directly to Vocab with this (or other) language. We have been having considerable recent discussions on extensible binding, though, and there is some considerable sentiment from some to change the extensible rules and guidance. Some of the use cases and considerations being discussed as reasons for doing that I think have some of the US Core guidance and experience in mind as examples, or at least it's describing the issues in a similar way, but that connection is only indirect. I think I can say with a high degree of certainty that in Vocab we have never approved the US Core language - but I also have to say that I don't know what would have happened if we had actually addressed it.
John Moehrke (Jun 27 2021 at 15:50):
if the source only has text, like from a legacy environment that allowed data entry as simple text... then how is it to do current deterministic and perfect code matching? -- Might these cases be something that we recognize as not compliant with us-core, while recognizing that they might be returned from a search? Is this what they are trying to do? To try to pull these results that are clearly not capable of being compliant into somehow seeming to be compliant, while not compliant... this very much devalues an IG compliance claim. Better that everyone recognize that some search results will not be us-core compliant. Which begs the question, is there some indication of all the results that are compliant vs not? Surely profile tagging is possible.
Eric Haas (Jun 27 2021 at 17:05):
Implementers are providing feedback that a strict interpretation of extensible can not be met. They also expressed strong reservations about investing in a mapping exercise that adds little or no value. US Core's approach is pragmatic and addresses these real world concerns. My primary concern is that a strict adherence to modeling rules will be an unnecessary burden and therefore a barrier to further adoption of US Core. My secondary concern is that after 5 months of discussions and ballot reconciliation, following the HL7 balloting processes,openly discussing all these issues including the contentious ones, striving to provide straightforward clear guidance where the practice may not toe the line with the spirit of the FHIR Specifications, documenting these changes better that anybody else, and only now on the cusp of publication this issue being brought up to derail it.
Grahame Grieve (Jun 27 2021 at 19:52):
It's not being brought up to derail it, it's being brought up to ensure it doesn't get derailed later. Lloyd hasn't expressed any issue with what you've agreed, only how you expressed it. Of course, the challenge is that what you'e agreed is bound up in how you expressed it, and changing that would send you back to square one on the agreement.
And this is not the only non-compliance, so it's not clear to me that it's a prima facie reason to derail the process. And @Lloyd McKenzie it's not like the rules around extensible are that water-tight. it's always been an argument and a matter for human interpretation
Lloyd McKenzie (Jun 27 2021 at 23:03):
The arguments about Extensible have been about it not doing what people would like it to do, not so much about what the words in the spec clearly say. The reality is that 'Extensible' is useful - as defined - but only works when you're talking about net new data. You're essentially saying "you must code data if it fits in this space and you must code it this way if the codes provided cover the concept". That's a legitimate (and desirable) thing to say. And it can be essential to drive interoperability. If you relax the constraint, then anyone can send whatever the heck they feel like and interoperability goes away.
The problem is that Extensible isn't appropriate for what US Core is trying to do - which is essentially "please make your best efforts". But that's really hard to conformance test.
I'm totally fine with US Core relaxing their constraints to whatever they think is achievable. I'm not comfortable with them saying that those rules constitute what Extensible means.
Richard Townley-O'Neill (Jun 28 2021 at 05:00):
It seems US Core wants to use preferred bindings for legacy data and extensible bindings for new data. That seems to want a pair of profiles.
Brett Marquard (Jun 28 2021 at 13:23):
The issue is that the main EHR implementers object to sending something like a generic SNOMED code for 'procedure' when they're sending free text or a local code that they don't have an ability to translate.
Not true. They object because their customers see no value in sending 'high-level' procedure code. They could hard code procedure and be 'conformant' ...
Brett Marquard (Jun 28 2021 at 13:28):
I haven't going through my notes but it definitely feels like we have discussed US Core extensible binding in every ballot. There are several zulip chains + trackers. Pretty extensive chat on this topic on Implementers March 2.
Brett Marquard (Jun 28 2021 at 13:39):
While I don't think it will happen for this publication -- it would be great to have a fully blessed approach for 'we sometimes have free text'. It may be as simple in the future as saying sometimes instance will be 'non-conformant' and that is ok.
Brett Marquard (Jun 28 2021 at 13:59):
...then of course systems will default high-level concepts to be conformant.
Josh Mandel (Jun 28 2021 at 14:28):
Agreed -- summarizing my take:
-
Sending high level concepts in place of specific codes doesn't help implementers, even if it's "valid". We should steer away from the unhelpful / trivially valid.
-
US Core doesn't intend to apply by-the-core-spec extensible binding to old/externally sourced data
-
Generating errors or warnings on poorly coded data is important, which is one motivation for the US Core profiles to apply extensible binding even though it's not technically accurate -- leveraging tooling!
I think @Richard Townley-O'Neill 's summary is right in descriptive terms, but if you provide an "escape hatch profile" it might become much harder to raise awareness about data quality gaps.
Lloyd McKenzie (Jun 28 2021 at 15:50):
I'm happy to steer away from trivially valid - but I don't want to steer toward technically invalid.
I think the issue is that regulation doesn't provide an out for old/external data, so there's a "need" for that old/external data to be 'valid' against US Core. Question - are there any impositions that US Core seeks to impose on legacy/external data? And do they have a clear definition of what constitutes legacy/external such that tight conformance testing could still be performed on 'new' data?
I don't think multiple profiles makes it harder to raise awareness about quality gaps. You can still validate against the 'tight' profiles (and perhaps soften errors to warnings).
Eric Haas (Jun 28 2021 at 16:22):
Businesses would and perhaps should , if given this choice of profiles, always choose the loosest one.
Lloyd McKenzie (Jun 28 2021 at 17:17):
They're going to do that regardless of expression mechanism if you don't set rules for when certain profiles must be used. That's a problem you have with the current approach too. (In addition to being non-conformant with the core spec :smile: )
Richard Townley-O'Neill (Jun 29 2021 at 10:56):
While writers of data may want to do the minimum, readers may prefer more.
Businesses that want to process the data can test against the tight profile to see how hard it will be to process the data.
They can even whinge about sub-standard data with an objective standard of quality.
Lloyd McKenzie (Jun 29 2021 at 12:30):
You're correct. One of the key uses of profiles is to tell client apps what they can count on when dealing with the data. That's why Extensible exists - so that an app can look at code or text it doesn't and understand and be able to at least reason "I know this isn't one of the concepts in the value set". If the source system can't guarantee that it either doesn't expose the instance or doesn't declare the profile.
Profiles aren't expected to apply to all data. The problem here is that US Core is trying to make everything fit in one profile - which results in a profile that can't enforce anything and that clients can't rely on for anything.
Grahame Grieve (Jun 29 2021 at 12:46):
Profiles aren't expected to apply to all data.
Well, there's no much other choice in this case. And clients can't rely on anything... that's the real world, I believe.
Lloyd McKenzie (Jun 29 2021 at 13:19):
Certainly the real world is that all instances won't comply with the desired constraints - but the methodological solution to that is "not all instances will comply with the profile". It's not to create a profile that covers the lowest common denominator of all instances. At least not if you expect the profile to have much value - either from setting a target or giving clients data they can count on.
Lloyd McKenzie (Jun 29 2021 at 13:21):
More importantly, in environments where profiles are being used to say "this data follows the desired rules", Extensible functions exactly as you'd like it to.
Last updated: Apr 12 2022 at 19:14 UTC
