Stream: implementers
Topic: Extensible bindings
Grahame Grieve (Jun 01 2018 at 06:30):
I've posted a note on my blog about the proper use of extensible bindings here: http://www.healthintersections.com.au/?p=2810.
tl;dr version:
- if you're not working on an expensive government project, don't use the extension valueset-reference (see http://build.fhir.org/extension-valueset-reference.html)
Robert McClure (Jun 01 2018 at 18:27):
If you send a code not in the value set bound extensible and you (per Graham's post) do not indicate what value set this code did came from, then No One, including the receiver can say what value set the code came from. IE: it's not from the extensibly bound value set.
Lloyd McKenzie (Jun 01 2018 at 19:03):
"no one can say what value set it's from" means you can't presume that it's not from the extensibly bound value set. It could be from any value set.
Lloyd McKenzie (Jun 01 2018 at 19:04):
Because most of the time the value set isn't declared it's not because it's from some special value set, it's because the sender didn't bother to share what value set it was associated with - that's the typical behavior of most systems.
Lloyd McKenzie (Jun 01 2018 at 19:06):
There is no mechanism you can use that will allow reliable automated terminology validation of extensibly bound data elements. You can tell if something from within the value set was used. But the only way to check if something from outside the value set is "legal" is to have a human look at it.
Robert McClure (Jun 02 2018 at 20:07):
The critical point here is that, as Graham has noted, it's the binding that is extensible, not the value set so sending something not in the value set means it must have come from some other value set, if someone needs to know that.
Lloyd McKenzie (Jun 02 2018 at 20:19):
Technically the code chosen doesn't need to have been from a value set at all, though most computer systems wouldn't support users directly entering a code without some sort of value set. (And of course, no code at all is also a possibility.)
Robert McClure (Jun 04 2018 at 23:21):
The point as I understand it is that If I ask the sender, where is the concept from, they would have to tell me what value set I can find it in. To be clear, I'm not saying I think that is best, it's just what I'm understanding. I know they already will have provided a url/uri for the code system but as we all know, there is no guarantee that means anything so bad-actors can simply send a made up url for a made up code system and any "concept" they want to send. By allowing for a situation where receivers can expect testable conformance, the only way to do that is with a value set.
Grahame Grieve (Jun 04 2018 at 23:31):
they can make that up too. Whatever arrangements you put in place to prevent that apply to the code system too
Lloyd McKenzie (Jun 05 2018 at 02:03):
Receivers cannot have testable conformance with Extensible bindings. Period. Senders will send codes that aren't recognized. And there's no computable way of verifying that the concept being conveyed couldn't be covered with one of the approved codes. Imposing a requirement to declare if you're using the approved value set is an unnecessary cost as it's easy enough to tell that the code falls within the value set. Requiring the declaration of some other value set (which the receiver doesn't have access to) when someone sends something outside the approved value set doesn't really help. It means you know the sender knows it's not in the approved value set. It doesn't tell you what's important to know - which is whether it's a concept that couldn't be sent within the approved value set.
Robert McClure (Jun 05 2018 at 20:19):
@Lloyd McKenzie You have actually had multiple discussions with me arguing extensible is testable given smart implementers like you :-) But assuming you are correct that Extensible is not testable, then ipso-facto, the only testable strength is Required. @Grahame Grieve do you agree, or can you tell us how using http://build.fhir.org/extension-valueset-reference.html you can specify an IG that can be conformance tested?
Lloyd McKenzie (Jun 05 2018 at 20:25):
There's testable by automated means and there's testable by humans. If I have an extensible binding of "red", "green", "blue" and an instance that says {"code"="00315", "system"="http://foo.bar"}, adding a value set declaration into it doesn't help you computably figure out if 00315 is legal or not. If 00315 means "light blue" or "emerald green", then it's invalid. If it means "white" or "safety orange", it's valid. And no value set declaration (or absense there-of) is going to tell a computer that. A human has to look at it.
Lloyd McKenzie (Jun 05 2018 at 20:27):
That said, if it's not computably conformance testable, it's not computably conformance testable. You don't make something required on the grounds you want conformance testing. You make something required on the grounds that you have a set of codes that completely cover the space and you know no-one's ever going to need to say something that can't be covered by the value set you've got.
Lloyd McKenzie (Jun 05 2018 at 20:28):
If you don't have that, then you have no choice but to be Extensible or looser. And if that means that you can't computably conformance test every instance, so be it. You can still computably test most of them. And, if you wish, you can flag the exceptions for human review.
Robert McClure (Jun 05 2018 at 20:32):
I think this highlights that you are interested in model alignment but you don't believe we can do semantic interoperability. Most systems map to exchange terminologies anyway so who is to say what was meant by the person entering that data versus what is in the exchange object. We have to use words in a meaningful way and "human testable" is pretend testing IMHO - it's not useful and provides dangerous belief that "truth" has been found. We do need to say when something is and can be computably conformance tested and there is a requirement for to be done by implementers. You seem to think that is a one-off on a case by case basis. I certainly hope that is not the case.
Lloyd McKenzie (Jun 05 2018 at 20:38):
Lets take a specific example - marital status codes. It has an extensible binding. It has that binding because we believe that the set of codes is well enough defined and small enough that systems throughout the world can map most, if not all of their concepts to the value set we've defined. But we also know that in such an space with the intertwining of legal and cultural expectations, there's a pretty decent likelihood of additional concepts being needed in at least some jurisdictions at some point. That's reality. As much as we might like to have the power to say to the world at large "you shall not define any additional types of mariage-like personal relationships", we don't. That doesn't mean we can't do semantic interoperability. The extensible binding to the set of codes we've got will give us really good semantic interoperability in 99%+ of systems and circumstances. If that's as good as we can get, that's as good as we can get.
Lloyd McKenzie (Jun 05 2018 at 20:40):
For something like procedure codes, extensibility is going to be even more important. We can't possibly have a valueset that will encompass every procedure that will exist for all time, even if we tighten ourselves to a narrow space like "Canadian human denistry".
Lloyd McKenzie (Jun 05 2018 at 20:41):
Healthcare evolves. Our standards have to allow for that. Allowing for evolution means that testing is harder. Not allowing for evolution just means that systems will choose to be non-conformant because they have no choice - and as soon as you have that, the stigma around non-compliance and the ability to apply leverage to fight non-concompliance goes out the window.
Robert McClure (Jun 05 2018 at 22:55):
@Lloyd McKenzie , you keep defending something I agree with. I like extensible where we can use it. But I also need computable conformance testability PLUS extensibility. Tell me how to do both in a way that allows people to not define the value set that has the code not in the bound value set.
Lloyd McKenzie (Jun 05 2018 at 23:09):
I don't understand your last sentence. And how do you expect to achieve computable conformance testability in the situation where I send 00315 which might mean "light blue" or might mean "safety orange"?
Lloyd McKenzie (Jun 05 2018 at 23:11):
I understand you want both computable conformance testability + extensibility. I want to each cheesecake every day while getting down to 180 lbs. I might have slightly better odds.. ;)
Grahame Grieve (Jun 06 2018 at 03:07):
I also need computable conformance testability PLUS extensibility
Grahame Grieve (Jun 06 2018 at 03:07):
yes please explain what you mean by that
Grahame Grieve (Jun 06 2018 at 03:08):
do you mean, max value set binding?
Robert McClure (Jun 13 2018 at 21:27):
@Grahame Grieve Ok, I see that was confusing. I'm saying that the computable conformance testing can occur on the extensibly-bound value set, and with the "other" code including the source value set, then that value set can be "conformance tested" to at least confirm it was a real code from the value set and not an error. I am not suggesting that the computer can reliably confirm that the other code is indeed in scope and different from the bound value set - that requires a human.
Robert McClure (Jun 13 2018 at 21:35):
I have a related question that I'd like some clarity on: How different does a concept need to be to be considered a "proper" extensible code? I'd like there to be some concrete things said here. I'm worried that it could be as loose as "any concept that can be considered an ISA to a concept in the value set" should not be sent as extensible. For example, if we had a value set of vital signs with the code 29463-7 Body weight, is 3141-9 Body weight Measured an acceptable "extension" concept?
Grahame Grieve (Jun 13 2018 at 21:59):
I believe that in the past, vocab has said no.
Grahame Grieve (Jun 13 2018 at 22:01):
the idea here is that you know that the 29463-7 Body weight code, and that's all you need to know. Extensible codes are other things. if you turn around and say 'is-a' codes can be extensions, no you really haven't achieved anything at all with the extensible value set; no one can say that it means anything about the actual codes.... The max binding, if there is one, that's all that has meaning after all
Grahame Grieve (Jun 13 2018 at 22:01):
I think that's a pretty strong argument.
Rob Hausam (Jun 13 2018 at 22:09):
For the conformance testing, I think you're saying that you expect that the value set that the "other" code (outside of the extensibly bound value set) was obtained from will be sent on the wire - we can do that with the valueset-reference extension, but it's not required to do so and send over the wire (and it's also not necessarily "recommended" practice - you do it if and when you have a particular desire and need for it in your situation). You may have had the same intent in saying "with the "other" code including the source value set", but I wasn't clear about that.
Rob Hausam (Jun 13 2018 at 22:21):
For the "proper" extensible code, as Grahame is suggesting, what we've said is that the 'is-a' (subtype) codes wouldn't be allowed. I don't think that came originally from Vocab, but we haven't said otherwise. In the example of a "color codes" value set the idea would be that if the extensibly bound value set included the colors for 'red', 'blue' and 'green', you would be allowed to send a code for 'yellow' since its meaning isn't covered in the value set, but you couldn't send 'navy blue' because its meaning is covered by the code 'blue' (since 'navy blue' is a subtype). You would have to send the code for 'blue' in the instance - but you could also send an additional code for 'navy blue' (assuming that the element is CodeableConcept). I don't think the documentation on "extensible" in 4.1.4.2 has spelled that position out quite so clearly, though.
Lloyd McKenzie (Jun 13 2018 at 23:20):
I think that knowing that a code comes from "some" value set isn't a terribly useful thing. It doesn't tell you if the code is conformant with the spec. And if it doesn't tell you that, I don't see much point in expecting it to be present. (Particularly when I expect a miniscule percentage of systems to bother declaring value sets at all)
Grahame Grieve (Jun 13 2018 at 23:22):
see my blog post - it has a fractional usefulness in highly validating environments. Part of that usefulness is where the validation doesn't stop at 'there's a value set reference' - you can also validate the value set reference itself because that must be declared, and you have it's provenance.... now we've moved into an entirely different league that is starting to make it useful and maybe even machine validatable.
Lloyd McKenzie (Jun 13 2018 at 23:25):
Knowing that some code is part of a value set doesn't mean it's valid against an extensible binding from some other value set. You can only tell that computably if the code is from the same code system or has sufficient mappings that tell you that it's orthonal to the concepts that are in the extensible-bound value set. And you don't need valueset declarations to do that.
Lloyd McKenzie (Jun 13 2018 at 23:26):
I can send an arbitrary code and I can send you a value set that says "all codes from that code system". What have I accomplished in terms of computable validation?
Grahame Grieve (Jun 13 2018 at 23:29):
well, it's about provenance. If I can make a rule that says 'the value set you must declare must be in registry X' and I control the entry to registry X, then I can say 'no, that's an invalid statement' and any statement that is valid is checkable
Lloyd McKenzie (Jun 13 2018 at 23:36):
Value set being in the registry isn't going to help unless the registry only allows value sets that are limited to concepts outside the Extensible value set. (Which seems a far-fetched scenario...)
Grahame Grieve (Jun 13 2018 at 23:38):
it's certainly far fetched right now because it hasn't been possible to set all this up. But certain large programs are thinking this way, and moving towards setting this up.
Lloyd McKenzie (Jun 14 2018 at 00:47):
The whole point of extensible is "We can't possibly know what all of the values are and we can't efficiently add them to the valueset at the time they're discovered". If you have the infrastructure to set up a registry of special value sets tied to an extensible value set that you're planning to use to validate, then why not make the value set required and have everyone register their new codes as they're needed? It'd be much more straight-forward...
Grahame Grieve (Jun 14 2018 at 00:52):
I think mainly I'd say that it become a governance question. I think i'd argue that you're right, but not everyone might be able to update so quick.
Rob Hausam (Jun 14 2018 at 00:55):
Right - but I think the point would be that if you know that you can't manage updates quickly enough to meet the needs then you also know that you're not going to be able to require validation of all of the codes.
Grahame Grieve (Jun 14 2018 at 00:57):
I agree with that. So we keep coming back to: the real world utility of the value set extension is non-zero, but still fractional
Grahame Grieve (Jun 14 2018 at 00:57):
maybe we should put a cost benefit assessment on extensions ;-)
Lloyd McKenzie (Jun 14 2018 at 01:10):
If you can't update so quick, then the code won't be in the registry value set either, and it won't validate.
Lloyd McKenzie (Jun 14 2018 at 01:11):
If people need to add codes faster than the registry can manage, you need extensible - and you can't validate without a human. If the registry can keep up, use required
Robert McClure (Jun 14 2018 at 16:40):
Gee, @Lloyd McKenzie really does dislike conformance testing in real-world systems ;-). Just so I'm clear, please explain how systems distinguish nonsense (entered in error) codes from codes intended to mean something? And also, how do "non-fractional" systems figure out if the code is an error, or is a proper extension code and what it means? Without a value set that would provide a description and perhaps even a definition. I want to point out that simply saying "the code system will tell you that" is not a valid statement in FHIR because there is no requirement to have a resolvable code system. So in Lloyd's world, extensible means send what ever you want, and I have no obligation to tell you want it means because I can make up an endpoint for the code system. If that is incorrect, show me how a system can, without making everything REQUIRED (I personally do not think that is an option), make sure the "extended code" is understandable to a human, let alone not an error.
Lloyd McKenzie (Jun 14 2018 at 16:55):
I like (and prefer) computable conformance testing. I just accept the reality that it's not always possible. Extensible does not mean send whatever you want. There are clear rules about what's allowed and what's not. But those rules are rarely going to be enforceable by automated systems - human review will generally be required.
The requirement for extensible codes is that they can't be subsumed by any of the concepts represented in the bound value set. If the codes sent happen to be from the same code system and that code system supports subsumption testing, then you're home free. For example, if the base value set is an enumerated list of SNOMED procedure codes and you send some other SNOMED code, it's pretty easy to test whether the code you sent is subsumable by any of the codes in the value set (thus invalid) or is not (thus valid, though possibly non-sensible). If you want to go beyond validity testing, the only way to check sensibility is to specify a max value set (which pushes you back into validatable) or to have a human review.
If the additional code doesn't come from the same code system, you'd have to have concept maps from whatever code system it does come from to the code system used for the extensible value set that would allow you to make subsumption (or at least equivalent/narrower) tests between that code and the codes from the approved value set. However, in most cases, extensible codes will, by definition, be local codes because the common case for extensibility is "procedures, diagnoses, medications, etc. that aren't in the official code system(s)". So when the next SARS comes along or someone invents an experimental medical procedure or you're dealing with a non-approved/registered drug, there won't be any knowledge about it that would allow you to computably make those inferences.
I don't really understand the "selected in error" case. In almost all user interfaces, the set of codes available to the user is going to be a dropdown/searchable set maintained by the local system. If you let a user pick codes without constraint, you have no hope of being compliant for an extensible value set. So the conformance determination is whether the "Extra" codes allowed in that system-specific value set are truly concepts that aren't subsumed by concepts in the bound value set. Having a value set identifier doesn't help you with that at all.
In answer to your question, "Understandable to a human" simply involves a conformance expectation that you send CodeableConcept.text and/or CodeableConcept.coding.display for any concept not in the bound value set. (Though, in general, good practice is to include those regardless.) In terms of "not an error", there are numerous types of errors and there are very few that can be detected computationally if the binding is extensible and no "probable" errors I can think of that would be mitigated by sending the identity of one or more value sets in the instance.
Last updated: Apr 12 2022 at 19:14 UTC
