Stream: questionnaire
Topic: Direct resolution of answerValueSet
Paul Lynch (Jun 25 2020 at 00:57):
If the FHIR server cannot provide an expansion of an answerValueSet's URI, the section on type canonical says that one should "fall back to direct resolution using the URL as a literal reference". Does this mean that the answerValueSet URI (as a URL) might be expected to return a ValueSet expansion or just a definition? If the latter, how does one expand it? By appending "/$expand"?
Brian Postlethwaite (Jun 25 2020 at 01:12):
That's what mine does, it appends the /$expand with filter=blah
Paul Lynch (Jun 25 2020 at 19:48):
What if the URL has a #fragment part? Would you then be expected not to $expand it, but to look inside the returned resource for .contained?
Paul Lynch (Jun 25 2020 at 19:50):
It would be helpful if there were a list of steps for how to process answerValueSet.
Lloyd McKenzie (Jun 25 2020 at 20:46):
You can't have a #fragment when pointing to a value set?
Brian Postlethwaite (Jun 25 2020 at 21:53):
Do you mean as a contained reference?
Brian Postlethwaite (Jun 25 2020 at 21:53):
In my renderer if it's contained I currently expect that to be pre-expanded.
Paul Lynch (Jun 25 2020 at 22:06):
Lloyd McKenzie said:
You can't have a #fragment when pointing to a value set?
It is just a more complicated thing at that point. You have https://example.com/SomeResource/123#fragment where "fragment" is now an ID of a ValueSet that is contained inside the SomeResource instance. So, instead of appending /$expand in the URL in that case, you would need to fetch https://example.com/SomeResource/123, and look inside .contained for ValueSet "fragment", which @Brian Postlethwaite says we should expect to be already expanded.
Brian Postlethwaite (Jun 25 2020 at 22:09):
I don't handle the url#fragment format. I thought you meant a contained reference. If that was in my renderer, I'd just append $expand, which is likely wrong.
Paul Lynch (Jun 25 2020 at 22:17):
Appending /$expand assumes that the answerValueSet URL is pointing to a server that not only can return the ValueSet, but can handle the /$expand operation. (Reasonable; I just think this should be make explicit in the answerValueSet field documentation.) The general page on canonical reference is helpful, but I think there needs to be something more specific on how to apply that when trying to get a ValueSet expansion for a Questionnaire.
Lloyd McKenzie (Jun 25 2020 at 22:18):
@Grahame Grieve - can you reference contained ids with a canonical URL with a fragment identifier? It may be possible with Reference, but I'm not sure it's allowed with canonicals...
Grahame Grieve (Jun 25 2020 at 22:24):
I don't know that we've discussed it?
Paul Lynch (Jun 25 2020 at 22:25):
From https://www.hl7.org/fhir/datatypes.html#canonical: "Like other URIs, elements of type canonical may also have #fragment references"
Lloyd McKenzie (Jun 25 2020 at 22:27):
Ok, that's super clear :)
Brian Postlethwaite (Jun 25 2020 at 22:27):
I expect we havent talked about the effect on operations, or explicitly expansions though.
Lloyd McKenzie (Jun 25 2020 at 22:30):
Do we need to add anything explicit into the SDC spec around this?
Brian Postlethwaite (Jun 25 2020 at 22:33):
I think we should.
Paul Lynch (Jun 25 2020 at 22:37):
The process for obtaining the answers from answerValueSet as I understand it is:
1) if the answerValueSet URI is like "#fragment", look for the ValueSet inside the Questionnaire's .contained, and assume it is expanded.
2) if the URI is like http://something#fragment, remove #fragment.
a) If a FHIR server is available, query that for http://something; Otherwise try to use the URI as a URL to get that resource
b) Obtain the ValueSet (assumed expanded) from the resource's .contained
3) If the URI is like http://something, If there is a "preferredTerminologyServer" set, query that for expansion of the answerValueSet URI; otherwise if there is a FHIR server available, try there; otherwise append /$expand and try to use the resulting string as a URL to get the expansion.
I have not thought about to how handle versions in the URI.
Another good part to document for answerValueSet would be to how get the expected size.
Lloyd McKenzie (Jun 25 2020 at 22:44):
@Brian Postlethwaite Change request? :)
Brian Postlethwaite (Jun 25 2020 at 23:08):
Sure.
Brian Postlethwaite (Jun 25 2020 at 23:08):
@Michael Lawley interested in your thoughts on this too.
Brian Postlethwaite (Jun 25 2020 at 23:09):
As a separate request we need to consider those chained questions where and answer in one changes the results of another questions valueset. We spoke about it, and I've forgotten what we considered to experiment with.
Brian Postlethwaite (Jun 25 2020 at 23:10):
I think that terminology handling will be a heading in the guide somewhere...
Michael Lawley (Jun 25 2020 at 23:31):
So my take would be slightly different I think.
- Is about resolving a relative URI/L, so first do that and hopefully then you have a local copy of an actual ValueSet.
- You have an absolute URI, so that's good.
Now, you either have an actual ValueSet or an Identifier for a ValueSet. In the former case, POST it to[base]/ValueSet/$expand. In the latter case, do a GET on[base]/ValueSet/$expand?url=[URI].
This assumes you have a remote ([base]) terminology server or a local capability, in which case these calls would be just routed internally.
Technically you're only meant to use a pre-expanded ValueSet if the expansion parameters match. Also, if your ValueSet expansion is large, then you're going to want a search-style UI rather than check box / pick list and that means you'll want to be calling $expand with the filter parameter.
Michael Lawley (Jun 25 2020 at 23:35):
I don't think you should be treating the URI as a URL and resolving it directly. It _might_ work, but relying on that is not a good idea.
Brian Postlethwaite (Jun 25 2020 at 23:45):
That's my irritation with canonicals, as they can be either.
Brian Postlethwaite (Jun 25 2020 at 23:56):
Thanks Michael, I guess that is a second alternative for the contained valueset resource, if it is expanded then you can use it directly with radios/checkboxes or a combo, if it isn't you need to post to get it to resolve.
That's correct @Michael Lawley ?
Paul Lynch (Jun 25 2020 at 23:58):
Michael Lawley said:
I don't think you should be treating the URI as a URL and resolving it directly. It _might_ work, but relying on that is not a good idea.
The current guidance for canonicals says we should do this as a fallback. "Systems resolving references to canonical URLs SHOULD first try to resolve the reference using the canonical reference (e.g. search on a known registry of terminology, conformance, or knowledge resources as appropriate), and then fall back to direct resolution using the URL as a literal reference if a local version of the canonical resource cannot be found."
Michael Lawley (Jun 26 2020 at 03:56):
Personally, I think that's a bad idea. We don't do it on principle because a malicious canonical might be used to probe local resources. eg file:///etc/passwd or http://localhost/..., https://intranet.acme.com/... because the resolution is happening server-side, inside the firewall
Brian Postlethwaite (Jun 26 2020 at 08:00):
That is great feedback Michael, and we should explicitly include that in the notes.
I may even reconsider my default implementation.
Paul Lynch (Jun 26 2020 at 12:48):
In my case resolution (for answerValueSet) happens client-side, in the user's browser. In the case of answerValueSet, where we are appending "/$expand", and expecting a ValueSet, I guess the security concern is limited to hitting some internal FHIR server (normally inaccessible) and loading a ValueSet from there into the Questionnaire. A maliciously crafted Questionnaire might then be able to send that data somewhere else using some of the other SDC extensions, without the user even submitting the form. I think maliciously crafted Questionnaires are already a potential problem, though. The app needs to be able trust the source of the Questionnaire.
Michael Lawley (Jun 27 2020 at 01:43):
In that case, similar security problems exist; a malicious Questionnaire could probe the user's systems either behind their home firewall/NAT router or behind their corporate firewall. Exfiltrating that data would require some scripting ability on behalf of the form runner - is this the "other SDC extensions" part?
Lloyd McKenzie (Jun 27 2020 at 01:53):
I don't really understand the threat. When you resolve the canonical, the target has to be a FHIR resource. If it's not, the system's not going to be able to parse it and expose it as a value set (or whatever it is you're referencing). The best you could do is probe whether a file exists by looking at whether you get a "not found" or "bad content" error - but you couldn't actually get the content
Michael Lawley (Jun 27 2020 at 02:36):
the system's not going to be able to parse it
this depends entirely on "the system", eg the form renderer. It may be that the renderer is very permissive in what it accepts, or it may be that it exposes the retrieved resource or fragments of it in an error dialog.
The exact risk depends on whether the fetch is happening server-side (so a hacker can use canonicals to probe behind the server's firewall, depending on how it manages and reports failures), or client-side (but this likely depends on the ability to inject active content into the questionnaire).
Paul Lynch (Jun 29 2020 at 12:45):
Lloyd McKenzie said:
I don't really understand the threat. When you resolve the canonical, the target has to be a FHIR resource.
Suppose the user's browser has access to an non-public FHIR server with private information, and an attacker knows this. If the attacker can get the user to load a Questionnaire containing canonicals pointing to that non-public server, the data could be loaded into the form and then sent to to the attacker.
Lloyd McKenzie (Jun 29 2020 at 13:57):
Only if the data was in the form of a ValueSet...
Lloyd McKenzie (Jun 29 2020 at 13:57):
Only if the data was in the form of a ValueSet...
Paul Lynch (Jun 29 2020 at 14:14):
In this case, yes, the canonical we are talking about is for answerValueSet, and so the expected data must be a ValueSet. It does seem less likely that sensitive data would be in a ValueSet. Perhaps there are other uses of canonical where resolving the URL directly might be more risky. There is a similar issue with x-fhir-query Expressions, which can be a full URL, and can pull back any sort of resource. I am not saying we shouldn't, but there are potential problems depending on the use case.
Jose Costa Teixeira (Jan 25 2021 at 20:54):
I hope this is the right stream/topic:
Jose Costa Teixeira (Jan 25 2021 at 20:55):
for the generation of a questionnaire form from a questionnaire resource, is there already a way to expand the answerValueSet at the time of IG publication?
Jose Costa Teixeira (Jan 25 2021 at 20:56):
I think the only way to do that now is either to use $expand somewhere (after sushi? in sushi?) or to do it manually and use answerOption
Paul Lynch (Jan 26 2021 at 01:08):
It sounds like you want to do that automatically as part of the IG publication process (with which I am not familiar). Why not just run $expand once, and include it in your Questionnaire as a contained ValueSet? Then answerValueSet can point to the contained copy (if that is acceptable).
Lloyd McKenzie (Jan 26 2021 at 04:50):
Are you asking for expansion of contained value sets as part of the publication process?
Jose Costa Teixeira (Jan 26 2021 at 08:24):
not only contained
Jose Costa Teixeira (Jan 26 2021 at 08:25):
I'm asking if the VS in a questionnaire are being / can be expanded in a way it can be used by a form generation tool
Jose Costa Teixeira (Jan 26 2021 at 08:27):
the expansion code is there, so I wonder if it's a good idea to preserve the expanded valueset for tooling.
Lloyd McKenzie (Jan 26 2021 at 14:48):
The publisher can and does expand value sets when rendering. Whether it should store the expansion in the published value set is a question we can ask. Keep in mind that the expansion at the time of publishing isn't necessarily what the expansion would be at the time the Questionnaire is used.
Jose Costa Teixeira (Jan 26 2021 at 16:59):
The expansion is only rendered, not preserved as a resource. We have several options - one of them would be to generate a Questionnaire that contains the expanded valueSets (for tooling that would be great, not sure if it is overall a sensible idea)
Lloyd McKenzie (Jan 26 2021 at 17:48):
My point is - why preserve the expansion if it it might change?
Jose Costa Teixeira (Jan 26 2021 at 18:07):
weak reason: tooling convenience.
Stronger reason: because it might change. You may want to produce the valusets as they are at a given time and package that in a single zip.
Lloyd McKenzie (Jan 26 2021 at 18:11):
If it's "you might want to", then it should be a configuration option - perhaps on a per value set basis.
Jose Costa Teixeira (Jan 26 2021 at 18:28):
yes, I was thinking it should not be default for every VS. Perhaps an extension on the VS that triggers that expansion?
Lloyd McKenzie (Jan 26 2021 at 18:56):
Submit a change request
Jose Costa Teixeira (Jan 26 2021 at 20:25):
change request to SDC or to FHIR spec?
If FHIR spec, a standard extension to VS?
Lloyd McKenzie (Jan 26 2021 at 20:48):
IG publisher, I think
Lloyd McKenzie (Jan 26 2021 at 20:48):
Would be relevant for value sets used in a variety of circumstances
Jose Costa Teixeira (Jan 26 2021 at 21:06):
standard extension expandWhenPublishing, coded.
Jose Costa Teixeira (Jan 26 2021 at 21:08):
codes = none | contained | resource
- none is default, do not expand
- contained creates the expanded valueset as a contained resource
- resource creates the expanded valueset as an autonomous resource
Jose Costa Teixeira (Jan 26 2021 at 21:08):
ok?
Lloyd McKenzie (Jan 26 2021 at 21:09):
Don't understand why it's not a boolean. The value sets might be contained or stand-alone. The expansion will live inside the value set, wherever it is.
Jose Costa Teixeira (Jan 26 2021 at 21:09):
replacing the valueset?
Jose Costa Teixeira (Jan 26 2021 at 21:09):
you'd have something in input and something different in output?
Jose Costa Teixeira (Jan 26 2021 at 21:10):
could be - because we are actually telling the system to expand, so I guess that is ok.
Jose Costa Teixeira (Jan 26 2021 at 21:11):
boolean was my first choice, I just thought that it would be better to allow expansion AND preserving the input as it is. Which sounds a bit silly... if the editor asks this to be expanded, then it gets expanded.
Lloyd McKenzie (Jan 26 2021 at 21:12):
You'd have an input with just compose and an output with compose + expansion
Jose Costa Teixeira (Jan 26 2021 at 21:12):
the only downside is that the published output should not be used as input in another IG...
Lloyd McKenzie (Jan 26 2021 at 21:12):
Why?
Lloyd McKenzie (Jan 26 2021 at 21:13):
If there was already an expansion and the extension was present, the expansion would get replaced
Jose Costa Teixeira (Jan 26 2021 at 21:13):
what woudl the publisher do? override the expansion ?
Jose Costa Teixeira (Jan 26 2021 at 21:13):
ok. I just want to see that all corners are covered.
Jose Costa Teixeira (Jan 26 2021 at 21:14):
ok, boolean and a warning in the extension definition that setting expandWhenPublishing to true WILL replace the existing expansion.
Lloyd McKenzie (Jan 26 2021 at 21:15):
I think that makes sense
Jose Costa Teixeira (Jan 26 2021 at 21:16):
is expandWhenPublishing a reasonable name?
Lloyd McKenzie (Jan 26 2021 at 21:27):
Works for me.
Jose Costa Teixeira (Jan 26 2021 at 21:43):
https://github.com/HL7/fhir-ig-publisher/issues/225
Jose Costa Teixeira (Jan 26 2021 at 21:44):
Jira for standard extension as well?
Lloyd McKenzie (Jan 26 2021 at 21:56):
No need
Jose Costa Teixeira (Mar 22 2021 at 13:43):
@Bryn Rhodes @Joe Amlung @Jonathan Payne
Bryn Rhodes (Mar 24 2021 at 02:35):
We've been working on a profile of Library called a "manifest" that would allow you to define expansion parameters for a set of related artifacts that all shared value set dependencies. That way you can say "I want this set of value sets expanded with these parameters and versions", and that statement can itself be a resource. We've developed it in the context of quality measures (and based on the notion of an expansion profile that the VSAC currently supports for this purpose), but the idea is generally applicable: http://build.fhir.org/ig/HL7/cqf-measures/measure-terminology-service.html
Bryn Rhodes (Mar 24 2021 at 02:35):
In particular, the $expand override and the $package operation.
Last updated: Apr 12 2022 at 19:14 UTC
