Stream: terminology
Topic: ValueSet.compose missing inactive/abstract?
Alexander Henket (May 30 2018 at 20:08):
Is there a reason why I cannot mark concepts in a ValueSet.compose as inactive or abstract, while this feature is supported in ValueSet.expand? I would have expected inactive and abstract on ValueSet.compose.ConceptSet.ConceptReference
Grahame Grieve (May 30 2018 at 20:25):
inactive and abstract are properties of a concept that a value set doesn't change
Alexander Henket (May 30 2018 at 20:39):
In DECOR we have had the notion for years that ValueSets can contain codes that have been in there in active state at some point, but which are currently discouraged for use.
This is not a property of the code system per se, although when the Code System deprecates a concept this is followed by the Value Set. By keeping them in the ValueSet you make their status explicit.
Just omitting them disallows the option of flagging that you are using a discouraged concept that may still be valid because it is a historic item.
Use case: our terminologists change their mind every once in a while about which LOINC or SNOMED CT is the right one. This does not affect status of the concept in LOINC or SNOMED CT, just the status of the concept in the ValueSet.
Alexander Henket (May 30 2018 at 20:43):
Same with abstract: abstract allows me to group concepts under a meaningful grouper concept that should not be used by itself. I can expand with those abstract concepts, but I cannot compose with those abstract concepts. Abstract or not is something defined by the Code System.
When I create an extensional value set I have the same use case for abstract as I would have for expansions of intensional value sets.
Grahame Grieve (May 30 2018 at 20:51):
I think that the first status is not 'inactive', it's something else. We don't have any thing for this at this time. Make a task for vocab....
Grahame Grieve (May 30 2018 at 20:52):
the second... I think it makes more sense to think of abstract as use case specific than concept specific, but HL7 has never done that, and it's not what the core principles say. So.. another task...
Alexander Henket (May 30 2018 at 20:56):
Tracker GF#17276 Support for marking concepts as inactive/discouraged at compose
Tracker GF#17277 Support for abstract in compose
Thanks
Peter Jordan (May 30 2018 at 21:03):
For an example of a ValueSet $expansion operation request that returns some inactive SCT Concepts, go to http://ontoserver.csiro.au/vstool/ , ensure that Ontoserver (STU3) is selected as a server and enter 'heart attack' in the search text.
Alexander Henket (May 30 2018 at 21:48):
Thanks Peter, but the expansion is all good. It's the compose I worry about.
Alexander Henket (May 30 2018 at 21:51):
extensional is the norm in my context. intensions (and thus expansions) are the exception.
Grahame Grieve (May 30 2018 at 21:57):
expansions should be your norm, irrespective of whether you have an intensional or extensional definition
Alexander Henket (May 31 2018 at 06:46):
True, but how do I create an expansion of an incomplete extensional definition meaning that I cannot convey abstract or inactive concepts? I cannot invent them upon expansion: they have to be there in the extensional definition for me to be able add them to the expansion. I'll work on creating a modifierExtension for our purposes, but my guess is that this would be better off being a core extension, if not a core element
Michael Lawley (May 31 2018 at 07:12):
Indeed - this is currently a gap in the (complete) mapping of SNOMED Reference Sets into ValueSets: inactive membership is not the same as an inactive concept
Grahame Grieve (May 31 2018 at 07:53):
so I don't believe it's a modifier extension, just a standard extension.
Alexander Henket (May 31 2018 at 07:58):
An extension may be ignored. You cannot ignore a concept being abstract or discouraged from use. Hence modifier to me.
Michael Lawley (May 31 2018 at 08:04):
It would need to be supported in both the CLD and the expansion
Grahame Grieve (May 31 2018 at 08:06):
read the definition of modifier carefully. The concept is in the value set - that is not being modified. You are providing extra information about it's use.
Alexander Henket (May 31 2018 at 08:07):
I'm providing information on the interpretation of the concept. I want validation errors for use of an abstract concept. I want validation warnings for use of a discouraged concept
Alexander Henket (May 31 2018 at 08:09):
Maybe... rethinking my own statement... discouraged might be an extension (not modifier). Abstract really needs to be in compose, for me to effectively create an expansion.
Alexander Henket (May 31 2018 at 08:11):
@Michael Lawley what's CLD?
Michael Lawley (May 31 2018 at 08:12):
CLD = Content Logical Definition (I think) -- basically the definition part of the ValueSet
Michael Lawley (May 31 2018 at 08:14):
Wouldn't the extension be modifying the meaning of a concept's inclusion in the ValueSet; it's included, but not intended to be used (hence my original idea of re-using abstract)
Alexander Henket (May 31 2018 at 08:16):
Yes I would expect, and noted that in the tracker, that I would need ValueSet.compose.include.concept.abstract comparable to ValueSet.expansion.contains.abstract
Lloyd McKenzie (May 31 2018 at 13:00):
isModifier is not about "cannot be ignored because it's important". isModifier is about "my understanding of the instance ignoring the element would be false". However, in this case, I believe that to be true. If you ignore "abstract" and see a code in a code system, your interpretation will be "this is a code I'm allowed to send over the wire". If the abstract flag is set, that interpretation is now false. So abstract is definitely a modifier.
Yunwei Wang (May 31 2018 at 13:45):
That is tricky. So a ValueSet can mark a concept as abstract without looking at corresponding code system?
Robert McClure (May 31 2018 at 15:26):
@Alexander Henket I'm trying to fully understand your use case. You are saying that you have a value set that when originally created, all the codes in the expansion are appropriate for use. Again, let's be clear that the CLD, the Compose part, likely will not have an enumerated list of codes to be included, it could be "all the descendants of Heart Disease." So at the start, all the codes in the expansion are appropriate for use, and are active in the code system. At some point in the future, some of these codes are no longer "appropriate for use in the context of the value set", but they are still active codes in the code system. Is that cortrect? And when this occurs, are you thinking it could be that the value set definition ("all the descendants of Heart Disease") has not changed, so no new version fo the definition? If true, then all this "new information" has to come from something that is provided outside of the CLD - some sort of code system supplement that $expand brings in and uses that is value set specific (IE: only used for this value set) - this would be an appropriate use of a supplement. Or, you could say that all this is managed inside the CLD (must be managed inside the CLD) and therefore would be some sort of extension (other?) that can be used in the compose structure. That would mean every time you needed to identify the specific "not appropriate" codes in the expansion, you need to create a new version of the value set and specifically "mention" each of these codes inside the CLD. And no matter what, this additional concept/code property/attribute would have to show up in the expansion.
I think we need to better understand the actual use case before we jump to a solution.
Alexander Henket (May 31 2018 at 19:14):
@Robert McClure What frequently happens, more frequently than I would like, is that terminologists change their mind on what concepts best represent what is needed. More than often this is a change of heart about one SNOMED CT concept or the other. In some cases we even switch from LOINC to SNOMED CT or local code system to SNOMED CT. This is all about value set and has no relation whatsoever with the status of concepts in the code system.
But yes, when things go obsolete in the code system that is also a reason for updating the value set, possibly retaining the old deprecated thing for backward compatibility.
For discouraged codes we don't distinguish 'reasons why'. A value set discouraged code and a code system deprecated code are marked the same.
In the terminology group I always hear people think that the CLD is usually intensional while in my experience it is not. You might argue that behind any extensional definition there is an intent and I'll give you that, but let me give you the numbers:
decor.nictiz.nl:
<valueSets
all="2683" extensional="2520" intensional="163"
valueSets_withDiscouraged="25"
valueSets_withAbstract="28"/>
art-decor.org:
<valueSets
all="3909" extensional="3660" intensional="249"
valueSets_withDiscouraged="4"
valueSets_withAbstract="242"/>
I've taken the latest version of each valueSet as starting point for "all"
Alexander Henket (May 31 2018 at 19:54):
Another terminology fact I'm not particularly proud of (I think) is the number of (mostly local) code systems we host, many of which have been defined through a value set as there is no formal expression of the code system, e.g. because the governing body never published anything but a webpage.
decor.nictiz.nl:
<codeSystems all="1193"/>
art-decorg.org:
<codeSystems all="1329"/>
So.... those extensional definitions may hold the only record in existence of a notion of abstract/deprecated
Robert McClure (May 31 2018 at 21:26):
@Alexander Henket I understand that many value sets are enumerated lists, but our solution must work for any value set type and not force everyone to do one particular definition approach. I also understand that new versions of a value set definition are driven by many needs. And finally I absolutely understand that, for a variety of reasons but primarily for historical queries, users need to be able to access concepts that were in a prior "appropriate for use" expansion that is generated using a prior version OR even using the same version but expanded using a different code system version (hence my concern about not focusing solely on enumerated definitions.)
I absolutely agree this need to have access to multiple expansion sets, so we can get concepts no longer in the value set, is a critical need. It's been discussed in a few places but no solution is clear. We also would benefit from knowing that a prior expansion concept was removed because it was wrong versus one that was appropriate at the time but is no available, or is no longer appropriate.
I don't think abstract is the right way to do this. I'd like to hear what others think could work. I do think adding the ability for $expand to return multiple expansions, or a union, or something when given a parameter that is intended to generate multiple expansions (date range?) is a start. Would something like that help?
Grahame Grieve (May 31 2018 at 21:27):
isn't this confusing abstract and inactive?
Grahame Grieve (May 31 2018 at 21:27):
abstract is about UI grouping?
inactive is about asking to remove things?
Michael Lawley (May 31 2018 at 22:45):
The problem with inactive is that it confuses the _concept being inactive_ (the commonly understood meaning) with the _concept's membership in the expansion_ being inactive.
abstract is not a good label for the latter either, except that the use-semantics (i.e., not user-selectable) is what we're after.
Currently the documented intent for abstract is:
codes designated by the underlying code system as not for use as a selectable concept in a particular context. These abstract codes are typically used as a grouping/searching mechanism, and can be included either by enumerating them, or by using a filter
This is a little strange since it says the code system decides abstract-ness, but that it is a context-sensitive property. context is generally unknown at the code system layer (that is what value sets are for, if I understand things correctly).
I'm not sure about the second sentence; that seems to be the general rule for any kind of code being included (except, perhaps, for _include whole code system_)
Grahame Grieve (May 31 2018 at 22:53):
i'm not sure what the second sentence is about either. But I think that abstract is not the same concept - it's serious and intended, but for structural presentation. there's no point having an abstract code in an expansion not generated for UI support, for instance.
Robert McClure (May 31 2018 at 22:53):
I'm still not sure why we want to put concepts in an expansion that are not for use. And remember if you change things in an existing CLD to now indicate a specific concept is "abstract" or whatever we end up calling it that is a new value set definition.
Michael Lawley (May 31 2018 at 22:54):
Also, I think there are two scenarios here. One, where the set of "deprecated" value set members is explicitly managed via the CLD, and another, which I'm not sure if people need, where there is a desire to know the set of all former members of the value set (either with respect to all previous versions of the value set, or all previous versions of the code system, or, most-likely, all previous versions of both).
For the first case, both Nictiz and SNOMED reference sets provide examples / use-cases.
Grahame Grieve (May 31 2018 at 22:54):
it's common to put them in to help users navigate a herirarchy - this is for UI purposes.
Grahame Grieve (May 31 2018 at 22:55):
but I think they're not really members of the value set per se, just things that get included in the expansion. We specifically say somewhere that ways to put these into the expansions are not provided in the spec; it's extension space (for now)
Robert McClure (May 31 2018 at 22:56):
I understand the use case for navigational codes that in some way the user needs to notified they are not "selectable", but that was not the use case for this request. We need to solve the use case, not changing codes for display on the fly.
Michael Lawley (May 31 2018 at 22:59):
I think the end use-case is dealing with historical data, commonly for validation or analytics / reporting.
Perhaps use of an explicitly maintained history ValueSet would be an appropriate solution?
Alexander Henket (Jun 01 2018 at 05:18):
@Robert McClure I don‘t get why we keep ending up discussing expansion. I’m totally fine with the properties available to me in .expansion.
My concern is in compose. Compose is missing abstract and inactive as present in expansion. I thought that abstract would be trivial and so I didn’t address that in my previous post.
I thought that we were only discussing different variations of ‘inactive’ of a concept.
So I would like to stay on topic which is how to properly represent enumerated lists in compose, so I can expand that in step 2. If the compose definition is good then expansion of an enumeratIon would be just a copy/paste from compose.
Robert McClure (Jun 01 2018 at 18:11):
@Alexander Henket I'm talking about expansions because that is the only value set object you can use to display/use concepts in an application. An expansion may not be a simple "paste of the compose" even when the compose is an enumeration of the individual concepts because of the parameters in $expand. I'll assume you know that so won't belabor it. As for the use case you described before when I asked for it, what you described should not be done with "abstract" or "inactive" so I'm not sure which topic you want to "stay on." Inactive has one meaning in code system space: Inactive concepts should be considered no longer available for entry into newly created data from the date that status was assigned. Inactive concepts may be present in value sets for use in historical queries. Active (and inactive) concepts marked as "abstract" generally means the concept is a navigational grouper of other non-abstract concepts and as such should not be used in a data record. People ignore this all the time and use of "abstract" is code system specific. Some code systems also mark active (that may eventually become inactive) concepts as "deprecated" which in general means, yes, this code is not intended for use, but if you must, it is allowed. Some code systems use this to notify users that the code will be inactive in a future version.
As you can tell, my discussion describes attributes of concepts in a code system. It is not describing some new attribute you are using the value set compose machinery to assign a new property to a concept. Is that what you want to do, take an existing value set definition, and then make a new one that adds new properties to enumerated codes in the compose, and then you want $expand on this new definition to produce the concepts with your value set-added property in the expansion?
Alexander Henket (Jun 01 2018 at 20:27):
If the definition cannot hold info for abstract or inactive: I cannot expand it properly. If the definition can hold that info: I can expand. So let's concentrate on compose so I can expand properly later.
As long as the compose part of ValueSet cannot give me or lead me to abstract or inactive: there is no way that any expansion parameter can help me.
Abstract in compose would do the exact same thing in compose for me as is defined on expansion. I don't see a reason to discuss that use case.
Inactive in compose would do the exact same thing in compose for me as is defined on expansion. Sure it is meant to signal a concept being inactive in the code system, but intention is the same when I deactivated in the ValueSet: I'm telling you to preferably not use the concept. So although the reason for me telling you that might be different: the net result is the same.
Maybe keeping it small: given this composition A B C from code system CS for which there is no CodeSystem resource. Most value sets have that problem.
How would I expand that composition to: A (abstract), B (normal concept), C (inactive)?
Grahame Grieve (Jun 01 2018 at 20:32):
why is there no code system? better to keep your use case to snomed CT, and where the value set author wants to say things beyond what is in snomed about status
Grahame Grieve (Jun 01 2018 at 20:32):
firstly, with abstract: do you have any use cases where you want to have an abstract code in the expansion where it doesn't have children?
Grahame Grieve (Jun 01 2018 at 20:33):
note these words from the spec:
An expansion may include entries in the expansion that only serve an arbitrary grouping purpose, to make it easier for a human to use the list. These entries have no system or code, and must be marked as abstract. Note that the CodeSystem resource and ValueSet.compose offer no support for defining hierarchies and groups, but this does not exclude servers from using extensions or other knowledge to introduce such groups as an implementation featur
Grahame Grieve (Jun 01 2018 at 20:34):
with regard to abstract, you are exploring the thing we offer no support for.
Grahame Grieve (Jun 01 2018 at 20:34):
e.g. extensions....
Grahame Grieve (Jun 01 2018 at 20:35):
inactive is slightly different, I think. I look at that and say, why not have more than one value set?
Alexander Henket (Jun 02 2018 at 06:05):
Abstract (like specializable) always groups something. So it shall have 1..n children but shall not be picked itself. Note that the v3 code systems all have that notion and the abstract concepts are part of the code system there. See https://www.hl7.org/fhir/v3/ActCode/cs.html for example. I guess the property notSelectable translates to abstract=true in an expansion.
<concept>
<code value="_ActAccountCode"/>
<display value="ActAccountCode"/>
<definition value="..."/>
<property>
<code value="notSelectable"/>
<valueBoolean value="true"/>
</property>
Alexander Henket (Jun 02 2018 at 06:07):
The way v3 used it is exactly how we use abstract
Alexander Henket (Jun 02 2018 at 06:35):
If we were to use intensional value set definitions, then upon expansion I would go back to the code system and ask it for information on abstract. I could then expand properly without that information being in compose.
However: I have extensional definitions. Sometimes for code systems that the governing body never created anything like a code system for, or not something I can call on (web pages with listings, web search forms etc.). In these cases there is nothing but the ValueSet to tell me what I want/need. Here I need definitional support in the ValueSet for abstract.
Second reason is just plain performance: if have a notion of abstract in the extensional definition, then I can spare me the step of going back to a code system if available.
Alexander Henket (Jun 02 2018 at 06:44):
why is there no code system? better to keep your use case to snomed CT, and where the value set author wants to say things beyond what is in snomed about status
I hope I answered the first part. The use case around "inactive" (discouraged or whatever the name might be if inactive doesn't fit) is more applicable to SNOMED CT than abstract (I think).
Inactive/discouraged is only relevant when I want a concept in a ValueSet for backward compatibility reasons only. Maybe because I use that ValueSet for validation purposes and I don't want to error on historic data.
Discouraging a concept is for use is a ValueSet statement. Reason for discouraging use might be deprecation in the CodeSystem, but could also be because another concept suits the requirements better for the pick list that the ValueSet is supposed to support, even if the original concept in the CodeSystem is still active.
Grahame Grieve (Jun 02 2018 at 06:54):
so since there is no support for heirarchy in compose, I don't believe that the right thing for you to do is to introduce an extension on compose.concept for abstract. Do you have multiple levels?
Alexander Henket (Jun 02 2018 at 06:59):
Ha! I missed not having a hierarchy. Yes: in order to group, one must have hierarchy. Abstract groups by design
Alexander Henket (Jun 02 2018 at 07:01):
So another ticket then?
Grahame Grieve (Jun 02 2018 at 07:03):
well, you can, but it's not going to go anywhere for R4. I suggest that you introduce an extension that defines a display and what concepts are in it.
Alexander Henket (Jun 02 2018 at 07:03):
Alternatively I would need to forgo the idea of a ValueSet CLD and just keep expansions. That is what I originally started out with.
Grahame Grieve (Jun 02 2018 at 07:03):
that's for abstract. inactive...
Alexander Henket (Jun 02 2018 at 07:04):
Some expansions would thus have a definition of type "unable to reconstruct"
Alexander Henket (Jun 02 2018 at 07:10):
I suggest that you introduce an extension that defines a display and what concepts are in it.
I could invent a level extension for example to infer grouping from, but I will be only one that understands how that works upon expand. Conformance resources in general have less to gain from extensions because there would be no tooling support for them.
Grahame Grieve (Jun 02 2018 at 07:15):
that doesn't necessarily follow. If I think that the extension makes sense, I'll implement it, and I expect it would make it's way into connectathons.
Alexander Henket (Jun 02 2018 at 07:15):
Anyway. What I think I'll do from here
I'll use inactive for discouraged and abstract for exactly that in expansions. Inactive definition doesn't fit my use case, but the net effect is that people will be inclined to pick inactive for anything new and that is what I wanted so pragmatically I'm ok.
For the definitional part I'll try to figure out extensions, and work with Firely to get rendering for them on Simplifier. No idea if they'll fall for that. It would really help if they were core extensions for that.
If core extensions on compose are still something worth considering then I'll keep the tickets open otherwise I'll withdraw my two tickets.
Alexander Henket (Jun 02 2018 at 07:16):
Posts crossed each other
Grahame Grieve (Jun 02 2018 at 07:16):
I think we'd consider core extensions. I'd certainly recommend the committee talk about them
Alexander Henket (Jun 02 2018 at 07:17):
Thanks, should I flag In Person on them? I normally never do that
Grahame Grieve (Jun 02 2018 at 07:20):
do you usually join vocab calls?
Rob Hausam (Jun 02 2018 at 12:25):
It would be great to have @Alexander Henket join a Vocab call (or calls) to discuss this. The time is probably manageable for Europe at 21:30 - 23:00 on Thursday (and we could slot this for the early part of the call if needed). The next call is the 14th.
Michael Lawley (Jun 04 2018 at 07:22):
@Alexander Henket can you explain your plan again? I'm confused about your references to inactive. Specifically, I can't tell whether you're referring to ValueSet elements or CodeSystem elements.
Alexander Henket (Jun 04 2018 at 07:26):
Sure. I'll try to make myself available next week.
Alexander Henket (Jun 04 2018 at 07:35):
@Michael Lawley inactive is an element in ValueSet defined for signaling inactivation of concepts in CodeSystems. What I want does not fit that definition, but.... the reason for signaling in a ValueSet is probably because you want to tell implementers that they should not use that particular concept. And the latter is exactly what I want.
So I want use inactive in ValueSet expansion for signaling "discouraged for use" regardless of CodeSystem concept status. It seems like a 'good enough fit'. We can discuss further next week.
Michael Lawley (Jun 04 2018 at 07:44):
Ah, ok. That seems risky; I would have though abstract was a better choice for that hack - it's inherently valueset-specific
Michael Lawley (Jun 04 2018 at 07:44):
And it really does mean 'do not use'
Grahame Grieve (Jun 04 2018 at 07:45):
abstract is a different kind of thing; I think it makes perfect sense to be able to differentiate between 'don't use because it's a grouping concept' and 'don't use because we're trying to get people to stop using it'
Alexander Henket (Jun 04 2018 at 07:45):
I only want to discourage, not forbid. That makes abstract a lesser fit
Michael Lawley (Jun 04 2018 at 08:23):
interesting...I just noticed the disclaimer at the end of ValueSet.expansion.contains.inactive
It might not be known or specified whether an concept is inactive (and it may depend on the context of use).
This suggests that it is not just a pass-through from CodeSystem.concept.property.code="inactive"
Alexander Henket (Jun 04 2018 at 08:24):
Well all the better for me then :-)
Robert McClure (Jun 04 2018 at 23:06):
@Alexander Henket Just because someone wrote that does not mean it's a license to run amok ;-). Looking forward to next weeks discussion.
Grahame Grieve (Jun 14 2018 at 04:05):
see task GF#16814: just proposed a disposition for this task, for block vote, that resolves the abstract (but not inactive) task isse
Michael Lawley (Jun 14 2018 at 05:06):
I've just added a comment indicating how the proposed solution doesn't help for the historical-members use case
Grahame Grieve (Jun 14 2018 at 05:14):
well, i said it didn't resolve that use case
Grahame Grieve (Jun 14 2018 at 05:15):
because that use case is the 'inactive' use case not the 'abstract' use case
Michael Lawley (Jun 14 2018 at 07:36):
Ah, yes, I missed that distinction
Grahame Grieve (Jun 14 2018 at 07:38):
I'll review the rest of the tasks that exist, and if I don't find something, I'll create it
Grahame Grieve (Jun 14 2018 at 07:42):
ah GF#17276 - made a proposed disposition for that too
Michael Lawley (Jun 14 2018 at 12:01):
Yep, that gets deprecated members (which are still members, but discouraged).
The case I have (from SNOMED) is former-members. However, having thought about this a little more I wonder whether it might be better to address this via an additional implicit ValueSet pattern -- something like .../fhir_vs=refset-inactive/{sctid} ? This could then be dealt with in the SNOMED on FHIR group
Grahame Grieve (Jun 14 2018 at 12:14):
I don't mind defining implicit sct value sets as well if that helps someone
Last updated: Apr 12 2022 at 19:14 UTC
