Stream: questionnaire
Topic: Summary Scoring of Multiple Questionnaires
Grey Faulkenberry (Sep 07 2021 at 13:29):
Trying to decide the best way to summarize multiple questionnaires. For instance, if I have a set of 4 10-question questionnaires. They each have subscores and total scores for the questionnaire. My plan was to turn each of these into an observation. However, there are two other scores I need to calculate. I need the total scores of all of the questionnaires as a single sum. Then I have a complexity score. This is calculated from different questions in each of the questionnaires. For instance, I may need question 1-2 from questionnaire #1, question 3-4 from questionnaire #2, no questions from questionnaire #3 and then questions 5-10 for questionnaire #4. Is there a recommended way to specify these scores? Should I create another questionnaire with calculatedValues? Or is there a better resource (DiagnosticReport or Observation for instance) that I should be using?
Lloyd McKenzie (Sep 07 2021 at 13:34):
I would say Observation with a derivedFrom pointing to the relevant Questionnaires would be fine. Do you need to surface the specific values used in the calculations?
Grey Faulkenberry (Sep 07 2021 at 13:59):
Not for the total sum. We're going to create a report that will contain the complexity score and the total scores from all of the questionnaires. Then lower down in the report, we're going to have the Totals from each of the questionnaires, then the subscores from each of the questionnaires, and finally the answers to the individual items.
Grey Faulkenberry (Sep 07 2021 at 14:04):
If I used Observation with derivedFrom then the logic would have to be specified outside of FHIR?
I've been trying to create this as flexible as possible. So currently, we're going through the workflow that you and I have discussed previously (overall Task, sub-tasks for each Questionnaire, etc). So that using the Task, we know which questionnaires to display, and then from that I can dynamically create a score and Observations from the Questionnaire, with all of that logic specified in the FHIR resources.
Grey Faulkenberry (Sep 07 2021 at 14:08):
Is there a similar way to specify the complexity score logic (different questions from different questionnaireResponses), to be embedded in the FHIR resources, so that anyone that can render and work with FHIR could use them?
Lloyd McKenzie (Sep 07 2021 at 15:04):
The logic could be specified in a PlanDefinition or Library. @Bryn Rhodes may have some specific guidance. The main reason that Questionnaires embed logic is so that the scores can be calculated and displayed as the user fills them in or because scores may influence what gets displayed or when the Questionnaire is deemed 'complete'.
Bryn Rhodes (Sep 07 2021 at 18:24):
You could use a Measure to produce the score, that would be a continuous variable measure with logic that references the QuestionnaireResponse resources for each Questionnaire to get the component scores.
Grey Faulkenberry (Sep 11 2021 at 14:51):
@Bryn Rhodes So I've been thinking about this, and I think I like the measure idea. As I said, I currently have a set of questionnaires. Each questionnaire has some subscores and a total score. There is also a separate score that's calculated from certain questions in each questionnaire. I had included FHIRPath expressions for the subscores, but I'm not actually using them for display purposes. So then I thought about taking out all of the FHIRPath expressions, and moving them all, along with the combined score calculation, to a single measure resource, would that make sense?
Bryn Rhodes (Sep 13 2021 at 15:33):
Hi @Grey Faulkenberry , yes, that would make a lot of sense, especially if the questionnaire-specific calculations are just part of calculating the overall score.
Grey Faulkenberry (Sep 13 2021 at 16:49):
Awesome. Any chance you have any examples of such a thing? No worries if not, I just always find examples helpful when I'm trying to build something out.
Grey Faulkenberry (Sep 13 2021 at 19:52):
Sorry, I have to ask for some more guidance. I was thinking about something vaguely like this:
Grey Faulkenberry (Sep 13 2021 at 19:52):
{
"resourceType": "Measure",
"scoring": {
"coding": [
{
"system": "http://terminology.hl7.org/CodeSystem/measure-scoring",
"code": "continuous-variable",
"display": "Continuous Variable"
}
]
},
"compositeScoring": {
"coding": [
{
"system": "http://terminology.hl7.org/CodeSystem/composite-measure-scoring",
"code": "opportunity",
"display": "Opportunity"
}
]
},
"group": [
{
"id": "Scores for Questionnaire1",
"population": [
{
"criteria": {
"language": "text/fhirpath",
"expression": "%response1.item.answer.valueCoding.extension.value.aggregate($this + $total, 0)"
}
}
]
},
{
"id": "Scores for QuestionnaireA",
"population": [
{
"criteria": {
"language": "text/fhirpath",
"expression": "%responseA.item.answer.valueCoding.extension.value.aggregate($this + $total, 0)"
}
}
]
},
{
"id": "Summary Score",
"population": [
{
"criteria": {
"language": "text/fhirpath",
"expression": "%response1.item.where(linkId = '/questionnaire1/1' or linkId = '/questionnaire1/4').answer.valueCoding.extension.value.aggregate($this + $total, 0)%responseA.item.where(linkId = '/questionnaireA/3' or linkId = '/questionnaireA/8').answer.valueCoding.extension.value.aggregate($this + $total, 0)"
}
}
]
}
]
}
Grey Faulkenberry (Sep 13 2021 at 19:54):
Although I'm not sure, do I include both the scoring and compositeScoring concepts, since I think I'm using both?
Grey Faulkenberry (Sep 13 2021 at 19:56):
And then my goal was to get the total of all the answers for Questionnaire1 as the first score, the total for all answers in QuestionnaireA, and then the sum of questions 1 & 4 in Questionnaire1 and 3 & 8 in QuestionnaireA.
Lloyd McKenzie (Sep 13 2021 at 21:20):
@Bryn Rhodes
Bryn Rhodes (Sep 13 2021 at 22:53):
Just threw together a quick example using the PHQ9 depression questionnaire from the spec:
Bryn Rhodes (Sep 13 2021 at 22:54):
This library shows how to calculate the score from questionnaire responses:
Bryn Rhodes (Sep 13 2021 at 22:54):
https://github.com/cqframework/sample-content-ig/blob/master/input/cql/PHQ9Logic.cql
Bryn Rhodes (Sep 13 2021 at 22:54):
This library shows how to pull those together (from potentially multiple questionnaires, just add more libraries for the different questionnaires:
Bryn Rhodes (Sep 13 2021 at 22:54):
https://github.com/cqframework/sample-content-ig/blob/master/input/cql/QuestionnaireMeasure.cql
Bryn Rhodes (Sep 13 2021 at 22:55):
And this Measure resource shows how to express that as a continuous variable measure:
Bryn Rhodes (Sep 13 2021 at 22:55):
Bryn Rhodes (Sep 13 2021 at 22:55):
There's a test case for the PHQ9Logic library but it doesn't currently run, I suspect because it's hitting QuestionnaireResponse (which we don't typically do in Quality Measures) but I need to debug on that further.
Grey Faulkenberry (Sep 15 2021 at 19:07):
@Bryn Rhodes and @Lloyd McKenzie Alright, final question (I hope), for now. Some of the logic of how I create the MeasureReport is not fully defined in FHIR, but the output should be. Would you mind taking a glance at an abbreviated MeasureReport I'm generating to see if there are any issues that you notice?
{
"resourceType": "MeasureReport",
"group": [
{
"code": { "text": "Scoring for Developmental Milestones: 23-28 months" },
"stratifier": [
{
"stratum": [
{
"value": {
"text": "Total Score for Developmental Milestones: 23-28 months"
},
"component": [
{
"code": {
"coding": [{ "code": "Review", "display": "Review" }]
},
"value": {
"coding": [
{
"system": "mayjuun.com/fhir/ValueSet/low_medium_high_risk",
"code": "high-risk",
"display": "high risk"
}
]
}
}
],
"measureScore": { "value": "10" }
}
]
}
]
},
{
"code": { "text": "Scoring for PSC-17: Preschool" },
"stratifier": [
{
"stratum": [
{
"value": { "text": "PSC-17: Preschool, Irritability Subscore" },
"measureScore": { "value": "9" }
}
]
},
{
"stratum": [
{
"value": { "text": "PSC-17: Preschool, Inflexibility Subscore" },
"measureScore": { "value": "3" }
}
]
},
{
"stratum": [
{
"value": { "text": "PSC-17: Preschool, Routines Subscore" },
"measureScore": { "value": "6" }
}
]
},
{
"stratum": [
{
"value": { "text": "PSC-17: Preschool, Total Score" },
"component": [
{
"code": {
"coding": [{ "code": "At Risk", "display": "At Risk" }]
},
"value": {
"coding": [
{
"system": "mayjuun.com/fhir/ValueSet/low_medium_high_risk",
"code": "high-risk",
"display": "high risk"
}
]
}
}
],
"measureScore": { "value": "18" }
}
]
}
]
},
Bryn Rhodes (Sep 24 2021 at 21:38):
@Grey Faulkenberry , apologies for the delay, that looks good, yes
Grey Faulkenberry (Sep 24 2021 at 21:48):
@Bryn Rhodes no worries for the delay, thanks for the input!
Grey Faulkenberry (Nov 10 2021 at 20:02):
@Bryn Rhodes I had another question about MeasureReports. I've successfully implemented multiple questionnaires that are scored together using one Measure and produces a MeasureReport of multiple scores (total and sub-scores) from the different questionnaires. Is there a way to reference the specific questionnaire (or QuestionnaireReport) that the MeasureReport is scoring? Is that only available through the evaluatedResource?
Bryn Rhodes (Nov 10 2021 at 20:29):
Hi Grey, what context are you wanting to do this in? Are you processing the MeasureReport post evaluation?
Grey Faulkenberry (Nov 10 2021 at 22:27):
Yeah. So the workflow is that an outside clinician/social worker/someone, enters a request (which comes into our system as a ServiceRequest). That ServiceRequest triggers the creation of a Task. The inputs to the task are Questionnaires and a Measure from a PlanDefinition (based on age). We take those questionnaires from the task, download and render them for the end user. When the end user has completed all of the Questionnaires (there's a bunch of them - too many in my opinion but what do I know? - although we are allowing them to save their progress and return to it later), but when they've all been completed, we use the Measure to score all of the Questionnaires and create the MeasureReport. The MeasureReport (currently) is going to then be used to create a PDF (again, not necessarily the way I would have gone), but this way we can present the final report to Clinicians who use an EHR that we don't have access to. And they asked if there was a way I could create a reference to the questionnaire or questionnaireReport within that MeasureReport to indicate where the data was coming from.
Bryn Rhodes (Nov 10 2021 at 23:46):
Yes, the evaluatedResources for an individual MeasureReport will have references to the data used to compute the measure, so that would include the QuestionnaireResponse resources, and those will in turn have a reference to the Questionnaire.
Grey Faulkenberry (Nov 11 2021 at 17:44):
Hmmmm, ok. So there's no way to associate a specific score with a specific QuestionnaireResponse?
Grey Faulkenberry (Nov 11 2021 at 17:46):
For instance, if I have 4 scores, each from a separate questionnaire, I'll have the list of scores, and then a list of Questionnaires. But is there a way to show that the first score came from the first questionnaire, the second came from the second, but then scores 3 & 4 both came from the third Questionnaire?
Bryn Rhodes (Nov 11 2021 at 17:59):
In the CQL, you could pretty easily return the Questionnaire with the score. Do the scores have a LOINC code associated? (Or even a code from your IG if they don't have LOINC codes), you could construct Observation resources to hold the calculated score and the associated LOINC code. You could also use the derivedFrom element of the Observation to reference the QuestionnaireResponse.
Grey Faulkenberry (Nov 11 2021 at 18:01):
They don't have LOINC codes, these questionnaires were created for this project (although they are some of them part of the SWYC assessments).
Grey Faulkenberry (Nov 11 2021 at 18:02):
So far I've been avoiding creating new Observations and have just left the MeasureReport as the resource that we're storing the scores in
Grey Faulkenberry (Nov 11 2021 at 18:04):
(also I've been using FHIRPath to perform the calculations)
Grey Faulkenberry (Nov 11 2021 at 18:05):
Currently my MeasureReport still looks pretty similar to the one I posted earlier in this conversation if that's at all useful for illustration purposes
Bryn Rhodes (Nov 11 2021 at 18:06):
In that case you could create codes in the IG where you are creating the Questionnaires, and use that. If you don't want to put it in an observation, you could put it in the "group.code" element for the score.
Grey Faulkenberry (Nov 11 2021 at 18:07):
Aha, I like that, thanks!
Last updated: Apr 12 2022 at 19:14 UTC
