Stream: questionnaire
Topic: SDC Extraction examples
Elliot Silver (Feb 01 2021 at 21:31):
Do we have any examples of what a Questionnaire, QuestionnaireResult and result of an $extract look like for each of the extraction types? I'm looking to do Structure-based extraction, but can't figure out what the Questionnaire needs to look like, or what a compliant QR would look like.
Jose Costa Teixeira (Feb 01 2021 at 22:05):
I'm also interested in this - the documentation in SDC helps, but I'd like to know if there is any set of working examples, or any implementation out there
Oliver Egger (Feb 01 2021 at 22:35):
i don't think you would need any specific requirements on the QR for a structure map based extraction. I'm working on an example here: Questionnaire, Map and example for transform for QR
Paul Lynch (Feb 02 2021 at 13:05):
You can see the LHC-Forms output for Observation extraction by using our SMART on FHIR app at https://lhcforms.nlm.nih.gov/sdc. After selecting a patient, select the "Weight and Height Tracking Panel" under "Featured Questionnaires" on the left. Enter some values for weight and height, and then choose "Save As"->"FHIR QuestionnaireResponse (SDC) & Observations)". When the save completes, you will see a dialog box with links to the created QuestionnaireResponse and Observation records.
Elliot Silver (Feb 02 2021 at 17:43):
@Paul Lynch, thanks, I'll take a look at that.
Anyone have structure-based extraction?
Lloyd McKenzie (Feb 02 2021 at 19:21):
@Brian Postlethwaite
Brian Postlethwaite (Feb 02 2021 at 19:28):
I only have the definition based stuff, which I don't recall if is expected to be flunf in $extract.
Lloyd McKenzie (Feb 02 2021 at 19:30):
Oh, you mean StructureMap? @Joee Garcia
Elliot Silver (Feb 02 2021 at 19:31):
Oh, no, sorry, I meant definition-based. Or at least I'm hoping to use Definition; my expectation is that it will be simpler than StructureMap extraction.
Jose Costa Teixeira (Feb 14 2021 at 12:13):
@Paul Lynch I can't see the extraction working. I do choose a Save As and all I get is a QuestionnaireResponse
Jose Costa Teixeira (Feb 14 2021 at 12:16):
in the Definition-based extract, how does it work if we want to populate patient.identifier.value and observation.subject.identifier.value? What do we put in the questionnaire.item.definition?
Is there an example of how an extraction works?
Lloyd McKenzie (Feb 14 2021 at 15:38):
In theory, the QuestionnareResponse.subject should be copied to the Observation.subject - conveying reference, identifier, display, etc.
Jose Costa Teixeira (Feb 14 2021 at 15:40):
How does the system know this?
Jose Costa Teixeira (Feb 14 2021 at 15:42):
in the case above, where does the questionnaire.idem.definition point to? Patient.identifier? Or observation.subject.identifier?
Lloyd McKenzie (Feb 14 2021 at 15:42):
It's what we say towards the bottom of http://build.fhir.org/ig/HL7/sdc/extraction.html#designing-questionnaires-to-support-data-extraction
Lloyd McKenzie (Feb 14 2021 at 15:43):
We don't say to only copy part of subject, we say to copy the whole thing. We hadn't really talked about systems that only use .identifier instead of .reference, but there's no reason that wouldn't be copied too.
Jose Costa Teixeira (Feb 14 2021 at 15:44):
How do we say "don't create a Patient, just create a logical reference". or "create a patient, but contained"?
Lloyd McKenzie (Feb 14 2021 at 15:46):
We don't. My assumption is you'd do in the Observation the same as you had in the QuestionnaireResponse. If the QuestionnaireResponse had an external resource reference, there's no reason to have a contained or logical reference in the Observation
Lloyd McKenzie (Feb 14 2021 at 15:47):
Or more specifically, the use-case for doing so hasn't been raised or discussed.
Jose Costa Teixeira (Feb 14 2021 at 15:50):
Am I correct if I say that StructureMap extraction is more formal in the sense that in a StructureMap we really must have all of this covered?
Jose Costa Teixeira (Feb 14 2021 at 15:51):
(I think my misunderstanding was assuming that the Definition-based approach was usable out of the box, but it seems that there is interpretation and implementation work to be done)
Lloyd McKenzie (Feb 14 2021 at 17:55):
If you want to have super-strict control, StructureMap gives you that. Definition-based and Observation-based give less control, but are also less work. All are usable "out-of-the-box", so long as the assumptions built into the approaches work
Jose Costa Teixeira (Feb 14 2021 at 18:11):
But assuming structuremap provides something that can be implemented directly by the existing transformation tools, can I say that Definition-based approach is less work for interface designers, but more work for implementers?
Lloyd McKenzie (Feb 14 2021 at 22:12):
In either case, you need a library to do the work. The library to do arbitrary StructureMaps is harder to write than definition-based and much harder than Observation-based
Jose Costa Teixeira (Feb 14 2021 at 23:02):
Thanks.
I understood from @Oliver Egger that for structureMaps we could use a Transform operation.
Jose Costa Teixeira (Feb 14 2021 at 23:03):
but in any case, the library would then be the place to contain all that logic, right? handling references, knowing which resources to populate/create..
Elliot Silver (Feb 14 2021 at 23:48):
Lloyd McKenzie said:
We don't. My assumption is you'd do in the Observation the same as you had in the QuestionnaireResponse. If the QuestionnaireResponse had an external resource reference, there's no reason to have a contained or logical reference in the Observation
What I'm wondering is how, with the Definition-based approach (or, frankly any approach) , I can, say, create a Patient out of these items in the response; create 4 Observations from these items in spread throughout and interleaved in the response (including all using the same response element for, say, effectiveDateTime) and set the subject to the created patient; finally create an EpisodeOfCare to link all 5 together, based on additional response items? I just can't see how to go from the SDC documentation to implementing that. I expect that about half of the elements in each created resource needs to be based on hidden fixed response values, but beyond that, I'm lost.
Tadas Antanavicius (Feb 15 2021 at 01:40):
@Elliot Silver A few months ago I worked through a series of very similar (in complexity) use cases to the one you described, and tackled it with definition-based extraction. Indeed, it involved a lot of hidden response values and frankly extremely hard to read and reason about Questionnaires. We've since published an API to our implementation and maybe these examples will help: https://developer.commure.com/docs/apis/sdc/examples
But if I were to do it again, I'd jump to StructureMap based extraction as soon as you get into the complex logic of extracting multiple resources in a single Questionnaire, trying to link them, etc.
Elliot Silver (Feb 15 2021 at 02:19):
@Tadas Antanavicius Thank you for that insight, I think that is where @Jose Costa Teixeira and I were leaning. It _looks_ like Definition-based is easier, but my guess, (and by the sound of it, your experience) is that means you have to do a lot of work to keep from having to do work. :grinning:
Lloyd McKenzie (Feb 15 2021 at 03:24):
There's an operation for extraction generically - whether observation-based, definition-based or map-based.
@Elliot Silver For that level of complexity, you'd probably need to StructureMap and would spit out a transaction Bundle.
Definition works well when you're spitting out content that maps rather nicely to resources, but as soon as you're talking multiple layers of inter-related resources or the mapping involves any sort of transformation, StructureMap is the only thing powerful enough.
Oliver Egger (Feb 15 2021 at 06:06):
we are implementing the sdc extract and populate operation and hope to bring it to the next connectathon, if a form filler application will support calling these operation that would be cool!
Jose Costa Teixeira (Feb 16 2021 at 12:00):
Lloyd McKenzie said:
There's an operation for extraction generically - whether observation-based, definition-based or map-based.
If the same operation can handle the different options relying on the content of the Questionnaire/ QR, that'd be good.
Jose Costa Teixeira (Feb 16 2021 at 12:00):
and is this operation implemented somewhere on an existing server?
Jose Costa Teixeira (Feb 16 2021 at 12:00):
I'd love to use that to test some questionnaires
Jose Costa Teixeira (Feb 16 2021 at 12:22):
@Fred Hersch @Jing Tang
Paul Lynch (Feb 16 2021 at 14:35):
Jose Costa Teixeira said:
Paul Lynch I can't see the extraction working. I do choose a Save As and all I get is a QuestionnaireResponse
I am assuming you are referring to this:
1) https://lhcforms.nlm.nih.gov/sdc
2) Click "Try App"
3) Pick a patient
4) Click "Weight & Height tracking panel"
5) Fill in values for weight and height
6) Click "Save As"
There are two options under the "Save As" menu. The first, "FHIR QuestionnaireResponse (SDC)" only saves a Questionnaire. The second, "FHIR QuestionnaireResponse (SDC) & Observations" will both save the QR and do Observation-based extraction.
If you didn't get extraction, it might be that you didn't fill in values for weight and height, which are the items configured for extracting, or it might be that you chose the first "Save As" menu option, which doesn't extract.
Lloyd McKenzie (Feb 16 2021 at 14:58):
I'm not aware of anyone who's exposed the operation. It's possible some servers will only handle a subset of extraction types and will error out if you send one it doesn't recognize.
Jose Costa Teixeira (Feb 16 2021 at 16:42):
@Paul Lynch it does work, thanks. I guess I skipped some values before
Jose Costa Teixeira (Mar 19 2021 at 12:03):
two more people interested in a ready-available way to do $extract @Félix De Tavernier @Wei Zhao
Jose Costa Teixeira (Mar 19 2021 at 12:04):
is there any reference out there on existing implementations of the extract operation?
Jose Costa Teixeira (Mar 19 2021 at 12:04):
suppose that we want to a) write such code or b) use the FHIR validator for the StructureMap-based approach, or c)...
where would we start?
Oliver Egger (Mar 19 2021 at 12:11):
we will provide a $extract operation for the structure map base approached at the next fhir connectathon (@Alexander Kreutz is working on it). we can also publish the information here if we are ready before.
Jame Dang (Mar 22 2021 at 10:19):
@Oliver Egger : I'm a newbie in SDC Questionnaire, so I have a question : If we need to work with a complex questionnaire which cannot use the Observation base extraction so we can make an ad-hoc extraction (it means we extract data from the questionnaire response for only this question)? Thank you for your response
Oliver Egger (Mar 22 2021 at 13:26):
@Jame Dang I think that is a valid approach to extract the data directly from the questionnaire response
Lloyd McKenzie (Mar 22 2021 at 14:04):
You can choose to extract however you like. The SDC-defined extraction mechanisms are designed to produce either a full resource or Bundle of resources from a completed QuestionnaireResponse
Jame Dang (Mar 22 2021 at 14:11):
Yes. Thank you @Oliver Egger
Jame Dang (Mar 22 2021 at 15:40):
Thanks @Lloyd McKenzie too. I will try to do an ad-hoc for our case first. I tried to search but cannot find out any example for definition base and StructureMap based, for the Observation-based I found the application at https://lhcforms.nlm.nih.gov is enough good to use for the simple questions. Do you know any piece of code or application implemented the definition-based or StructureMap-based extraction?
Lloyd McKenzie (Mar 22 2021 at 15:43):
There's a StructureMap example in the Gravity IG. There have been a few examples posted here in the past for definition based, but we haven't got them incorporated in the SDC spec yet. @Brian Postlethwaite?
Jame Dang (Mar 22 2021 at 15:56):
Thanks @Lloyd McKenzie : I will look at that example but for our current work I think the ad-hoc extraction is best way? Our questionnaire is complex and the response need to convert to Patient, Observation, CarePlan and some other resoures.
Lloyd McKenzie (Mar 22 2021 at 15:59):
I'm not really sure what you mean by "ad-hoc" extraction
Jame Dang (Mar 22 2021 at 16:01):
It means I receive the QR and extract it case by case. For example I get the answer with the Patient ID then I will convert that to the Patient
Jame Dang (Mar 22 2021 at 16:03):
I mean It is the solutions for convert data from QR to other resources by the code for exactly our questionnaire. Is that a good approach?
Lloyd McKenzie (Mar 22 2021 at 16:08):
You can certainly write code specific to your Questionnaire - and that's probably going to be less effort than to handle generic conversion. However, you'll have to write custom code for each Questionnaire you want to be able to extract responses from, so if you expect to need to do it for a larger number of forms, it may be worthwhile to invest in a more 'standardized' mechanism.
Jame Dang (Mar 22 2021 at 16:17):
Yes. I understand that, but our current work is handling only 1 or some specific questionnaires and I think it not easy to do a 'standardized mechanism' in a short amount of time. I think we will try for the soecific case first after that we will try the more generic solution. Thank you very much for your advice
Jose Costa Teixeira (Mar 29 2021 at 21:47):
Looking for input: If the result of an extract operation is to be put in a Bundle, do we assume the Bundle
a) is a transaction
b) is a collection
c) needs to be defined as part of the extract mapping
Jose Costa Teixeira (Mar 29 2021 at 21:47):
Is there any recommendation on this?
Jose Costa Teixeira (Mar 29 2021 at 21:49):
I'd say a) seems a solid base. c) is more flexible but enough to make the mapping weird
Lloyd McKenzie (Mar 29 2021 at 22:19):
Generally a. However, agree this should be made more clear. Change request?
Jose Costa Teixeira (Mar 29 2021 at 22:54):
Will do
Oliver Egger (Mar 30 2021 at 06:52):
@Jose Costa Teixeira in our sdc dependent use case the result should be a document bundle, however if that would need to be wrapped into a transaction like a) that would be also possible.
Jose Costa Teixeira (Mar 30 2021 at 07:05):
Right, I hadn't considered a document, which makes sense
Jose Costa Teixeira (Mar 30 2021 at 16:05):
https://jira.hl7.org/browse/FHIR-31665
Richard Stanley (Apr 07 2021 at 18:18):
Hi folks, I'm trying to catch up with this thread and a bit confused. Is there an open source workflow to $evaluate ($extract?) a transaction bundle from QuestionnaireResponse using StructureMap?
Lloyd McKenzie (Apr 07 2021 at 20:42):
The FHIR validator includes a StructureMap processer, so it could be used. There's no wrapper that would let you just pass the QuestionnaireResponse and have it look up the Questionnaire and then the StructureMap from the extension
Ephraim Kigamba (Apr 30 2021 at 09:38):
is it guaranteed that sub-fields of the same field would always be grouped together? eg. Patient.name group question would ideally contain the given question and family question. Is this guaranteed to always be the case both for grouped questions and composite resource properties?
Paul Lynch (Apr 30 2021 at 12:55):
I think that is up to the person designing the structure of the Questionnaire.
Lloyd McKenzie (Apr 30 2021 at 13:16):
Nothing is guaranteed in the world of Questionnaire design :) However, population and extraction are easier when there's alignment. StructureMap can handle a great deal of misalignment. Definition-based population/extraction can handle a certain degree, but not unlimited.
Ephraim Kigamba (May 03 2021 at 08:34):
Thanks. I guess I'll research and see if I can come up with possible use-cases, examples and probably an easier way to define this.
Ephraim Kigamba (May 03 2021 at 10:08):
(deleted)
Ephraim Kigamba (May 03 2021 at 10:17):
For definition-based extraction, can we assume that if the group-item does not have a definition then the sub-group items are not type elements? Similarly, we can also assume that if the group-item has a definition, then the sub-group items are type elements?
I think based on the docs and the logic, this might make a lot of sense when considering alignment. This would help establish a "standard" to use where a resource or profile element can have multiple type values and we end up with multiple questions corresponding to the element. It would make it easier to align the values eg. a HumanName answer corresponds to index X of contact parties list in Patient.contact element where a patient has multiple contacts being registered
Lloyd McKenzie (May 03 2021 at 14:28):
A group not having a definition doesn't guarantee no elements on the children, but it would certainly be unusual. A group having a mapping suggests at least some of the child questions will have mappings, but certainly no expectation that all of them will.
Lloyd McKenzie (May 03 2021 at 14:29):
Keep in mind that most Questionnaires aren't designed based on FHIR alignment at all
Last updated: Apr 12 2022 at 19:14 UTC
