Stream: Covid-19 Response
Topic: Example Measure definition
Michael Donnelly (Mar 28 2020 at 15:28):
How does this look?
Michael Donnelly (Mar 28 2020 at 15:29):
{
"resourceType": "Measure",
"id": "erpDBndMnJbDcARBc8hK5iQ3",
"name": "MR COVID-19 SCREENING RATE",
"title": "Screening Rate",
"status": "draft",
"date": "2020-03-27T19:22:20Z",
"publisher": "Epic",
"contact": [
{
"name": "Michael Donnelly",
"telecom": [
{
"system": "url",
"value": "https://chat.fhir.org/#narrow/stream/226195-Covid-19-Response/topic/Example.20Measure.20definition",
"rank": 1
},
{
"system": "email",
"value": "michael.donnelly@epic.com",
"rank": 2
}
]
}
],
"description": "Percentage of patients with a face-to-face, telehealth, telephone, or admission encounter for whom a COVID-19 communicable disease screening was performed.",
"group": [
{
"population": [
{
"code": {
"coding": [
{
"code": "numerator",
"display": "Numerator"
}
]
},
"description": "Patients for whom a COVID-19 communicable disease screening was performed."
},
{
"code": {
"coding": [
{
"code": "denominator",
"display": "Denominator"
}
]
},
"description": "Patients with a face-to-face, telehealth, telephone, or admission encounter."
}
]
}
]
}
Josh Mandel (Mar 28 2020 at 15:33):
Having never really looked at a proper FHIR Measure before: this seems very clear to me, and I really like the telecon preferences :-)
Josh Mandel (Mar 28 2020 at 15:33):
I don't totally understand when you would use multiple groups in a measure versus defining multiple measures.
Michael Donnelly (Mar 28 2020 at 15:38):
Me neither. But I think the first time I actually looked at the Measure resource was this morning. So.
Michael Donnelly (Mar 28 2020 at 15:44):
I feel like there should be something in there to indicate this is COVID-19 related.
Michael Donnelly (Mar 28 2020 at 15:44):
Is that what useContext is for?
Michael Donnelly (Mar 28 2020 at 15:45):
None of the examples include it. :(
Josh Mandel (Mar 28 2020 at 16:44):
Yeah, unclear, but use context seems like a fine vet and at least it is searchable so If we pick a consistent code to populate there, you could ask a server to tell you about all the measures appropriate for use in covet.
Michael Donnelly (Mar 28 2020 at 16:47):
Like this?
Michael Donnelly (Mar 28 2020 at 16:47):
{
"resourceType": "Bundle",
"type": "searchset",
"total": 29,
"link": [
{
"relation": "self",
"url": "https://vs-icx.epic.com/Interconnect-CDE/api/FHIR/R4/Measure?context=covid19"
}
],
"entry": [
{
"link": [
{
"relation": "self",
"url": "https://vs-icx.epic.com/Interconnect-CDE/api/FHIR/R4/Measure/erpDBndMnJbDcARBc8hK5iQ3"
}
],
"fullUrl": "https://vs-icx.epic.com/Interconnect-CDE/api/FHIR/R4/Measure/erpDBndMnJbDcARBc8hK5iQ3",
"resource": {
"resourceType": "Measure",
"id": "erpDBndMnJbDcARBc8hK5iQ3",
"name": "MR COVID-19 SCREENING RATE",
"title": "Screening Rate",
"status": "draft",
"date": "2020-03-28T15:56:29Z",
"publisher": "Epic",
"contact": [
{
"name": "Michael Donnelly",
"telecom": [
{
"system": "url",
"value": "https://chat.fhir.org/#narrow/stream/226195-Covid-19-Response/topic/Example.20Measure.20definition",
"rank": 1
},
{
"system": "email",
"value": "michael.donnelly@epic.com",
"rank": 2
}
]
}
],
"description": "Percentage of patients with a face-to-face, telehealth, telephone, or admission encounter for whom a COVID-19 communicable disease screening was performed.",
"group": [
{
"population": [
{
"code": {
"coding": [
{
"code": "numerator",
"display": "Numerator"
}
]
},
"description": "Patients for whom a COVID-19 communicable disease screening was performed."
},
{
"code": {
"coding": [
{
"code": "denominator",
"display": "Denominator"
}
]
},
"description": "Patients with a face-to-face, telehealth, telephone, or admission encounter."
}
]
}
]
},
"search": {
"mode": "match"
}
},
{
"link": [
{
"relation": "self",
"url": "https://vs-icx.epic.com/Interconnect-CDE/api/FHIR/R4/Measure/e.ev-ttkAg3YKGZqfYcTc5g3"
}
],
"fullUrl": "https://vs-icx.epic.com/Interconnect-CDE/api/FHIR/R4/Measure/e.ev-ttkAg3YKGZqfYcTc5g3",
"resource": {
"resourceType": "Measure",
"id": "e.ev-ttkAg3YKGZqfYcTc5g3",
"name": "MR COVID-19 PRESUMPTIVE POSITIVE COUNT",
"title": "Total Presumptive Positive Count",
"status": "draft",
"date": "2020-03-28T16:00:22Z",
"publisher": "Epic",
"contact": [
{
"name": "Michael Donnelly",
"telecom": [
{
"system": "url",
"value": "https://chat.fhir.org/#narrow/stream/226195-Covid-19-Response/topic/Example.20Measure.20definition",
"rank": 1
},
{
"system": "email",
"value": "michael.donnelly@epic.com",
"rank": 2
}
]
}
],
"description": "The total number of patients for whom a presumptive positive result for a COVID-19 test was documented."
},
"search": {
"mode": "match"
}
},
{
"link": [
{
"relation": "self",
"url": "https://vs-icx.epic.com/Interconnect-CDE/api/FHIR/R4/Measure/eZYOIUgakUiwzbT1-tdRoVw3"
}
],
"fullUrl": "https://vs-icx.epic.com/Interconnect-CDE/api/FHIR/R4/Measure/eZYOIUgakUiwzbT1-tdRoVw3",
"resource": {
"resourceType": "Measure",
"id": "eZYOIUgakUiwzbT1-tdRoVw3",
"name": "MR COVID-19 PRESUMPTIVE POSITIVE COUNT (0-3 YEARS OLD)",
"title": "Total Presumptive Positive Count (0-3 YEARS OLD)",
"status": "draft",
"date": "2020-03-28T16:00:41Z",
"publisher": "Epic",
"contact": [
{
"name": "Michael Donnelly",
"telecom": [
{
"system": "url",
"value": "https://chat.fhir.org/#narrow/stream/226195-Covid-19-Response/topic/Example.20Measure.20definition",
"rank": 1
},
{
"system": "email",
"value": "michael.donnelly@epic.com",
"rank": 2
}
]
}
],
"description": "The total number of patients for whom a presumptive positive result for a COVID-19 test was documented. Only patients three years old or younger will be evaluated."
},
"search": {
"mode": "match"
}
},
{
"link": [
{
"relation": "self",
"url": "https://vs-icx.epic.com/Interconnect-CDE/api/FHIR/R4/Measure/eSdd7-fzTpLid3wVtWvjqPg3"
}
],
"fullUrl": "https://vs-icx.epic.com/Interconnect-CDE/api/FHIR/R4/Measure/eSdd7-fzTpLid3wVtWvjqPg3",
"resource": {
"resourceType": "Measure",
"id": "eSdd7-fzTpLid3wVtWvjqPg3",
"name": "MR COVID-19 PRESUMPTIVE POSITIVE COUNT (4-18 YEARS OLD)",
"title": "Total Presumptive Positive Count (4-18 YEARS OLD)",
"status": "draft",
"date": "2020-03-28T16:00:59Z",
"publisher": "Epic",
"contact": [
{
"name": "Michael Donnelly",
"telecom": [
{
"system": "url",
"value": "https://chat.fhir.org/#narrow/stream/226195-Covid-19-Response/topic/Example.20Measure.20definition",
"rank": 1
},
{
"system": "email",
"value": "michael.donnelly@epic.com",
"rank": 2
}
]
}
],
"description": "The total number of patients for whom a presumptive positive result for a COVID-19 test was documented. Only patients 4 to 18 years old will be evaluated."
},
"search": {
"mode": "match"
}
},
{
"link": [
{
"relation": "self",
"url": "https://vs-icx.epic.com/Interconnect-CDE/api/FHIR/R4/Measure/evA5yKQsGCRBBBYJAag08UA3"
}
],
"fullUrl": "https://vs-icx.epic.com/Interconnect-CDE/api/FHIR/R4/Measure/evA5yKQsGCRBBBYJAag08UA3",
"resource": {
"resourceType": "Measure",
"id": "evA5yKQsGCRBBBYJAag08UA3",
"name": "MR COVID-19 PRESUMPTIVE POSITIVE COUNT (19-44 YEARS OLD)",
"title": "Total Presumptive Positive Count (19-44 YEARS OLD)",
"status": "draft",
"date": "2020-03-28T16:01:19Z",
"publisher": "Epic",
"contact": [
{
"name": "Michael Donnelly",
"telecom": [
{
"system": "url",
"value": "https://chat.fhir.org/#narrow/stream/226195-Covid-19-Response/topic/Example.20Measure.20definition",
"rank": 1
},
{
"system": "email",
"value": "michael.donnelly@epic.com",
"rank": 2
}
]
}
],
"description": "The total number of patients for whom
Michael Donnelly (Mar 28 2020 at 16:47):
Whoops. Too big to fit in a single zulip post.
Michael Donnelly (Mar 28 2020 at 16:48):
But you get the idea.
Josh Mandel (Mar 28 2020 at 16:49):
Yes in that example I see the query parameter in the initial search, but I don't see the context in the resources return. Am I missing something?
Michael Donnelly (Mar 28 2020 at 16:49):
No, I am
Michael Donnelly (Mar 28 2020 at 16:49):
I didn't figure out how to display it yet (and I was working on something else.)
Michael Donnelly (Mar 28 2020 at 16:50):
Measure-search-example-v01.json
Michael Donnelly (Mar 28 2020 at 16:50):
There are two problems in there (at least - 2 I know of so far):
- it doesn't have the context (even though we know it)
- some of the descriptions had Epic-specific system stuff - I'm seeing if I can get rid of that programatically.
Josh Mandel (Mar 28 2020 at 16:53):
This is great. Are the measures here representing the things that Epic is collecting today, or the things in the CDC list, or the intersection?
Michael Donnelly (Mar 28 2020 at 16:53):
The former.
Michael Donnelly (Mar 28 2020 at 16:53):
Well
Michael Donnelly (Mar 28 2020 at 16:53):
They're the set that Epic specifically identified as being for COVID-19.
Michael Donnelly (Mar 28 2020 at 16:54):
We might have additional metrics that match ones in the CDC set that haven't been flagged yet with the COVID-19 context.
Josh Mandel (Mar 28 2020 at 16:54):
Cool! now that we have the first set from CDC it might be interesting to take the measures from their published collection form and represent in a similar way.
Michael Donnelly (Mar 28 2020 at 16:54):
Likely, actually.
Michael Donnelly (Mar 28 2020 at 16:55):
Makes sense.
Michael Donnelly (Mar 28 2020 at 16:55):
I still have some cleanup work to do on the Epic Measure resource, then I'm going to take a look at MeasureReport.
Josh Mandel (Mar 28 2020 at 16:55):
Awesome!
Michael Donnelly (Mar 28 2020 at 16:55):
(Also, Laura wants me to eat something, so that'll happen at some point.)
Josh Mandel (Mar 28 2020 at 17:01):
For UsageContext, https://www.hl7.org/fhir/measure-cms146-example.json.html has an example. I'll try to turn this into something related to covid19.
Josh Mandel (Mar 28 2020 at 17:08):
I think to indicate this is used in situations where the clinical focus is COVID-19, you'd do something like:
"useContext": [
{
"code": {
"system": "http://terminology.hl7.org/CodeSystem/usage-context-type",
"code": "focus"
},
"valueCodeableConcept": {
"text": "COVID-19",
"coding": [{
"system": "http://snomed.info/sct",
"code": "840535000",
"display": "COVID-19"
}]
}
}
]
Josh Mandel (Mar 28 2020 at 17:09):
And a search would be
Measure?context=http://snomed.info/sct|840535000if you have things neatly coded and up-to-date with the latest SNOMED release, orMeasure?context:text=COVIDfor a text-based search if you're not so sure
(I'm basing this on having just stumbled across this SNOMED page)
Michael Donnelly (Mar 28 2020 at 17:29):
I'm losing one of the systems for some reason.
Michael Donnelly (Mar 28 2020 at 17:29):
{
"resourceType": "Measure",
"id": "erpDBndMnJbDcARBc8hK5iQ3",
"name": "MR COVID-19 SCREENING RATE",
"title": "Screening Rate",
"status": "draft",
"date": "2020-03-28T16:47:55Z",
"publisher": "Epic",
"contact": [
{
"name": "Michael Donnelly",
"telecom": [
{
"system": "url",
"value": "https://chat.fhir.org/#narrow/stream/226195-Covid-19-Response/topic/Example.20Measure.20definition",
"rank": 1
},
{
"system": "email",
"value": "michael.donnelly@epic.com",
"rank": 2
}
]
}
],
"description": "Percentage of patients with a face-to-face, telehealth, telephone, or admission encounter for whom a COVID-19 communicable disease screening was performed.",
"useContext": [
{
"code": {
"code": "focus"
},
"valueCodeableConcept": {
"coding": [
{
"system": "http://snomed.info/sct",
"code": "840535000",
"display": "COVID-19"
}
],
"text": "COVID-19"
}
}
],
"group": [
{
"population": [
{
"code": {
"coding": [
{
"code": "numerator",
"display": "Numerator"
}
]
},
"description": "Patients for whom a COVID-19 communicable disease screening was performed."
},
{
"code": {
"coding": [
{
"code": "denominator",
"display": "Denominator"
}
]
},
"description": "Patients with a face-to-face, telehealth, telephone, or admission encounter."
}
]
}
]
}
Josh Mandel (Mar 28 2020 at 17:29):
Interestingly enough the example that I linked to is missing that system as well. I don't know why.
Josh Mandel (Mar 28 2020 at 17:30):
When you say you're losing it, do you mean you're running some software that is causing it to disappear from the output, when you've provided it as an input?
Michael Donnelly (Mar 28 2020 at 17:30):
Yes.
Michael Donnelly (Mar 28 2020 at 17:30):
But I have a hypothesis.
Josh Mandel (Mar 28 2020 at 17:30):
What software?
Michael Donnelly (Mar 28 2020 at 17:30):
Pilot Light.
Michael Donnelly (Mar 28 2020 at 17:31):
Epic's FHIR dev toolkit.
Michael Donnelly (Mar 28 2020 at 17:31):
(Danielle and I wrote it.)
Josh Mandel (Mar 28 2020 at 17:31):
Aha -- Cool! Google was failing me :-)
Michael Donnelly (Mar 28 2020 at 17:31):
Understandable. :)
Michael Donnelly (Mar 28 2020 at 17:33):
Fixed it.
Michael Donnelly (Mar 28 2020 at 17:34):
{
"resourceType": "Measure",
"id": "erpDBndMnJbDcARBc8hK5iQ3",
"name": "MR COVID-19 SCREENING RATE",
"title": "Screening Rate",
"status": "draft",
"date": "2020-03-28T16:47:55Z",
"publisher": "Epic",
"contact": [
{
"name": "Michael Donnelly",
"telecom": [
{
"system": "url",
"value": "https://chat.fhir.org/#narrow/stream/226195-Covid-19-Response/topic/Example.20Measure.20definition",
"rank": 1
},
{
"system": "email",
"value": "michael.donnelly@epic.com",
"rank": 2
}
]
}
],
"description": "Percentage of patients with a face-to-face, telehealth, telephone, or admission encounter for whom a COVID-19 communicable disease screening was performed.",
"useContext": [
{
"code": {
"system": "http://terminology.hl7.org/CodeSystem/usage-context-type",
"code": "focus"
},
"valueCodeableConcept": {
"coding": [
{
"system": "http://snomed.info/sct",
"code": "840535000",
"display": "COVID-19"
}
],
"text": "COVID-19"
}
}
],
"group": [
{
"population": [
{
"code": {
"coding": [
{
"code": "numerator",
"display": "Numerator"
}
]
},
"description": "Patients for whom a COVID-19 communicable disease screening was performed."
},
{
"code": {
"coding": [
{
"code": "denominator",
"display": "Denominator"
}
]
},
"description": "Patients with a face-to-face, telehealth, telephone, or admission encounter."
}
]
}
]
}
Michael Donnelly (Mar 28 2020 at 17:34):
(There's something weird when an element has the same name as its parent. )
Keith Boone (Mar 29 2020 at 05:37):
So, do I hear @Michael Donnelly signing up to do the MeasureDefinition for Epic's Measures? Anyone else want to sign up to take on the CDC set?
Keith Boone (Mar 29 2020 at 05:38):
@Hans Buitendijk does Cerner have definitions for what it's sending to CDC that could be turned into a MeasureDefinition resource?
Michael Donnelly (Mar 29 2020 at 10:56):
Yes. I’ve got all of them generating now, but I need to validate some cleanup I’m doing programmatically before I can share all of them.
Michael Donnelly (Mar 29 2020 at 10:57):
I’ll have the first set within an hour, but the rest will be another day or two.
Michael Donnelly (Mar 29 2020 at 12:20):
Measure-search-example-v02.json Attached is a Measure search bundle with a bunch of the measures Epic defined.
Michael Donnelly (Mar 29 2020 at 12:21):
I'm eager for feedback.
Josh Mandel (Mar 29 2020 at 14:12):
Overall these look really nice to me! I'm so pleased you're doing this. One comment about the different measures that address different age groups, like
Total Positive Count (4-18 YEARS OLD)
... have you considered defining a single measure for total positives and reporting out the different age groups as different strata, rather than defining different measures for each bucket?
Michael Donnelly (Mar 29 2020 at 14:50):
I did, in so far as I read a bit about it in the Measure spec and thought about it. Each of these Measure resources represents an actual record in Epic. Combining multiple records into one resource is definitely possible, but it'd add a lot of complexity, so it isn't high priority on my list right now.
Keith Boone (Mar 30 2020 at 02:40):
I wound't worry about combining them all right now, get them as individual resources. Someone can easily create a component to create a composite from the parts.
Hans Buitendijk (Mar 30 2020 at 02:54):
Just got the FEMA spreadsheet (not a link unfortunately). Will add that in the morning to gitHub to start a Test Summary wiki.
Hans Buitendijk (Mar 30 2020 at 13:59):
FEMA spreadsheet
State
County
New Diagnostic Tests Ordered/Received - Midnight to midnight cutoff, tests ordered on previous date queried
Cumulative Diagnostic Tests Ordered/Received - All tests ordered to date.
New Tests Resulted - Midnight to midnight cutoff, test results released on previous date queried
Cumulative Specimens Rejected* - All specimens rejected for testing to date
Cumulative Tests Performed - All tests with results released to date
New Positive COVID-19 Tests - Midnight to midnight cutoff, positive test results released on previous date queried
Cumulative Positive COVID-19 Tests - All positive test results released to date
Percent Positive among Newly Resulted Tests - # of new positive test results released / # of total new tests released for previous date queried
Cumulative Percent Positive among Resulted Tests - # of total positive results to released date / # of total tests results released to date
Michael Donnelly (Mar 30 2020 at 17:02):
@Gino Canessa
Gino Canessa (Mar 30 2020 at 17:07):
Thanks @Michael Donnelly !
Gino Canessa (Mar 30 2020 at 18:01):
@Michael Donnelly is there a reason for having Measure resources per age group as opposed to using a single Measure with groups and/or stratifiers in a single resource?
Carl Anderson (Mar 30 2020 at 18:30):
It was brought up and discussed briefly yesterday - it sounds like it would add some complexity on the Epic side because they already have the strata saved in separate records.
Michael Donnelly (Mar 30 2020 at 18:38):
What Carl said. :)
Michael Donnelly (Mar 30 2020 at 18:39):
I didn't write those Measures by hand; they represent metric records in the Epic DB.
Gino Canessa (Mar 30 2020 at 18:40):
Thanks, too many branches to follow.
I'm adding generators for these to my test data - I assume you'd prefer your name and email removed? :grinning:
Hans Buitendijk (Mar 31 2020 at 21:19):
Given that first round we are keeping the measures "simple", is "measure-observation" the right value for Measure.group.population.code? The other values do not seem to be right for the "simple" already stratified case.
Hans Buitendijk (Mar 31 2020 at 21:22):
Do we need Measure.contact.name for round one? I'm leaning to keep it optional, but not include it for now for the CDC and FEMA measures. Should get that at some point.
Keith Boone (Mar 31 2020 at 21:23):
Since measure.contact could be an organization, I think we can find something for CDC and FEMA, a website at least, until we get a real e-mail address.
Josh Mandel (Mar 31 2020 at 21:27):
I think for the "simple" version we don't have a group.population at all -- it doesn't convey information.
Josh Mandel (Mar 31 2020 at 21:28):
If we get to handling measures that include both numerator and denominator values, then we can add .populations for each.
Keith Boone (Mar 31 2020 at 21:30):
As for measure.group.population.code, There was a code for exactly this type of Measure in HQMF called MeasurePopulation
That's exactly what measure-population maps to. I would use that, it's technically accurate for what we are working on, means what the count means, and is aligned with prior work. measure-population does not have a denominator, in HQMF it required an initial-population, but this is not so in FHIR.
Keith Boone (Mar 31 2020 at 21:33):
As for not coding at all, since we have a code that means exactly what it is, I'd say used a fixed value for the Measure examples and MeasureReport examples, but don't profile it to being other than the https://www.hl7.org/fhir/valueset-measure-population.html value set. And for now, I'd say RESTRICT to that in the binding for this work. Let's not try to invent new types of things.
Hans Buitendijk (Mar 31 2020 at 21:34):
@Josh Mandel So following that for simple, should we then only require Measure.name, .title, .status, .date, .publisher, .description, and .guidance at a minimum and rest will follow?
Gino Canessa (Mar 31 2020 at 21:35):
Here's a basic version of what he's referring to (grabbed from a bundle.entry):
{ "fullUrl": "https://github.com/microsoft-healthcare-madison/learning-spike-erp/MeasureReport/FHIR-424", "resource": { "resourceType": "MeasureReport", "id": "FHIR-424", "status": "complete", "type": "summary", "measure": "http://cdcmeasures.example.org/modules/covid19/20200331/numbeds", "subject": { "reference": "Location/Loc-Org-1065" }, "date": "2020-03-31T16:31:08.0651578-05:00", "reporter": { "reference": "Organization/Org-1065" }, "period": { "start": "2020-03-28", "end": "2020-03-28" }, "group": [ { "measureScore": { "value": 9 } } ] } }
Gino Canessa (Mar 31 2020 at 21:36):
The matching Measure:
{ "fullUrl": "https://github.com/microsoft-healthcare-madison/learning-spike-erp/Measure/numbeds", "resource": { "resourceType": "Measure", "id": "numbeds", "url": "http://cdcmeasures.example.org/modules/covid19/20200331/numbeds", "name": "numbeds", "title": "Hospital Inpatient Beds", "status": "draft", "date": "2020-03-31T00:00:00Z", "publisher": "SANER-IG", "description": "Inpatient beds, including all staffed, licensed, and overflow(surge) beds used for inpatients.", "useContext": [ { "code": { "system": "http://terminology.hl7.org/CodeSystem/usage-context-type", "code": "focus", "display": "" }, "valueCodeableConcept": { "coding": [ { "system": "http://snomed.info/sct", "code": "840535000" } ], "text": "COVID-19" } } ] } }
Keith Boone (Mar 31 2020 at 21:39):
@Hans Buitendijk for contact info for FEMA, this fema-hhs-covid-diagnostics-tf@fema.dhs.gov is from the Pence letter to hospitals.
Hans Buitendijk (Mar 31 2020 at 21:39):
For Measure is where I'd suggest to add .guidance as well considering the additional clarifications being provided that go beyond description. But otherwise, o.k. with the above as starting point and not consider .group for CDC and FEMA measures at this point.
Michael Donnelly (Mar 31 2020 at 21:40):
I intend to have an example today.
Keith Boone (Mar 31 2020 at 21:41):
Why would we not consider group for this initial set?
Keith Boone (Mar 31 2020 at 21:45):
Also, this project has some (computable) names that are aligned with FEMA reporting: https://covidtracking.com/
The names are in their CSV format: https://covidtracking.com/api/us/daily.csv
Michael Donnelly (Mar 31 2020 at 22:25):
Michael Donnelly: Does that MeasureReport look like what you'd expect, @Keith Boone ?
Keith Boone : No, b/c the Measure itself is talking about COVID Virus, not vents. For vents you need a different code.
Moving this here, since it's about the Measure (and the other thread is about MeasureReport).
Keith Boone (Mar 31 2020 at 22:26):
706172005 | Ventilator (physical object)
257463002 | Ventilator outlet (physical object) (if counting slots)
Michael Donnelly (Mar 31 2020 at 22:26):
First, at this point I'm only pulling data from Epic's database. So I don't have any of our new codes or anything in there.
Michael Donnelly (Mar 31 2020 at 22:27):
I have the examples here to validate the form of the resources, and the content is to lend context to the form.
Michael Donnelly (Mar 31 2020 at 22:27):
No assertion that these are production ready; this is the rapid iteration part of the process. :)
Michael Donnelly (Mar 31 2020 at 22:27):
There'll definitely be things that aren't fully baked.
Michael Donnelly (Mar 31 2020 at 22:28):
Second, this Measure is definitely for ventilators:
{ "resourceType": "Measure", "id": "e8-H2kvdTuFUBWb2RX967fw3", "name": "ICU Total Ventilators", "title": "Total Ventilators", "status": "draft", "date": "2020-03-30T23:38:55Z", "publisher": "Epic", "contact": [ { "name": "Michael Donnelly", "telecom": [ { "system": "url", "value": "https://chat.fhir.org/#narrow/stream/226195-Covid-19-Response/topic/Example.20Measure.20definition", "rank": 1 }, { "system": "email", "value": "michael.donnelly@epic.com", "rank": 2 } ] } ], "description": "The number of ventilators available for use.", "useContext": [ { "code": { "system": "http://terminology.hl7.org/CodeSystem/usage-context-type", "code": "focus" }, "valueCodeableConcept": { "coding": [ { "system": "http://snomed.info/sct", "code": "840535000", "display": "COVID-19" } ], "text": "COVID-19" } } ] }
Michael Donnelly (Mar 31 2020 at 22:29):
The COVID-19 part in there isn't saying this is about a test for a virus; it's saying that the context in which we're using this measure is COVID-19 related reporting.
Michael Donnelly (Mar 31 2020 at 22:30):
Measure.title and Measure.definition say what we're measuring; useContext is why we're measuring it and reporting it right now.
Keith Boone (Mar 31 2020 at 22:30):
Oh, duh.
Keith Boone (Mar 31 2020 at 22:45):
Hans Buitendijk said:
For Measure is where I'd suggest to add .guidance as well considering the additional clarifications being provided that go beyond description. But otherwise, o.k. with the above as starting point and not consider .group for CDC and FEMA measures at this point.
I'm half with you. + 1 on guidance, -1 on not using Group, since we have the right way to do it, let's do it right. Group will show up soon enough, already needs to for some of FEMA which includes a proportion measure.
Josh Mandel (Apr 01 2020 at 00:14):
We have to use MeasureReport.group, but I'd suggest that we avoid using MeasureReport.group.polulation when we do not have any important semantics to express with it.
Keith Boone (Apr 01 2020 at 00:41):
What you might call important semantics and what others would call important semantics is likely different. I know I'm talking like "three lines of code" here, and in the current situation, we can ALL live without it. I'm thinking about the impact of this work is downstream, and the impact that missing 3 lines will have on the researcher who goes to use it... Since we know the right value and there's not a real argument over what it is, I'd like to include it in our examples.
Hans Buitendijk (Apr 01 2020 at 01:06):
@Keith Boone: Agreed that for FEMA Percent Positive among Newly Resulted Tests and Cumulative Percent Positive among Resulted Tests using .group.population for nominator and denominator makes sense as they have been defined that way. Not sure what would be meaningful for the others to put in .group as .description and .criteria.description would be the same as above. Agreed that measure-population could be used for .code upon further reading its definition and use.
Keith Boone (Apr 01 2020 at 02:24):
OK, so I've got my first example built, and you can see it on the most recent branch build here http://build.fhir.org/ig/AudaciousInquiry/saner-ig/branches/modelingForMeasureReport/Measure-SANERnumTotBeds.json.html , or below
{
"resourceType": "Measure",
"id": "SANERnumTotBeds",
"url": "http://ainq.com/fhir/us/saner/Measure/SANERnumTotBeds",
"version": "0.1.0",
"name": "numTotBeds",
"title": "Total Acute Beds",
"status": "draft",
"experimental": true,
"date": "2020-03-31T21:01:31-04:00",
"publisher": "Audacious Inquiry",
"contact": [
{
"name": "Audacious Inquiry",
"telecom": [
{
"system": "url",
"value": "http://ainq.com"
}
]
},
{
"name": "Keith W. Boone",
"telecom": [
{
"system": "email",
"value": "kboone@ainq.com"
}
]
}
],
"description": "Total number of all Inpatient and outpatient beds, including all staffed,ICU, licensed, and overflow (surge) beds used for inpatients or outpatients",
"useContext": [
{
"code": {
"system": "http://terminology.hl7.org/CodeSystem/usage-context-type",
"code": "focus"
},
"valueCodeableConcept": {
"coding": [
{
"system": "http://snomed.info/sct",
"code": "840535000",
"display": "COVID-19"
}
]
}
}
],
"jurisdiction": [
{
"coding": [
{
"system": "urn:iso:std:iso:3166",
"code": "US"
}
]
}
],
"topic": [
{
"coding": [
{
"system": "http://snomed.info/sct",
"code": "91537007",
"display": "Hospital bed, device (physical object)"
}
]
}
],
"relatedArtifact": [
{
"type": "documentation",
"url": "https://www.cdc.gov/nhsn/pdfs/covid19/57.130-toi-508.pdf"
}
],
"rateAggregation": "last day",
"guidance": "Enter the total number of all hospital beds, including\ninpatient and outpatient beds. All staffed, licensed,\nand overflow and surge/expansion beds used for\ninpatients or outpatients. This includes ICU beds.",
"group": [
{
"population": [
{
"code": {
"coding": [
{
"code": "measure-population"
}
]
},
"criteria": {
"language": "text/plain"
}
},
{
"code": {
"coding": [
{
"code": "initial-population"
}
]
},
"description": "All Beds",
"criteria": {
"language": "text/plain"
}
}
]
}
]
}
Keith Boone (Apr 01 2020 at 02:24):
I'm about to build the rest now that I have one working and tested.
Keith Boone (Apr 01 2020 at 02:25):
And then I'll work at organizing them into the guide in some way.
Keith Boone (Apr 01 2020 at 02:25):
I think the examples need their own section, with references to the FEMA and CDC Documents.
Bryn Rhodes (Apr 01 2020 at 02:27):
Looks great Keith, yes a url like that.
Bryn Rhodes (Apr 01 2020 at 02:28):
From a scoring perspective, are these continuous-variable measures?
Keith Boone (Apr 01 2020 at 03:18):
Yes they are measured like all continuous variables, and have a unit associated with them (the unit of thing being counted). They aren't measured in anything other than whole units though (at least for initial counts) though, so you won't see fractional units in typical reports.
HQMF has a good description of the types of measures, which I don't think we'll need to repeat, I'm just going to refer folks to QM and DEQM guides for that content.
Keith Boone (Apr 01 2020 at 03:19):
I'm going to push the other 21 measures to that branch and work on organization of the measures section of the guide.
Michael Donnelly (Apr 01 2020 at 03:22):
Josh Mandel said:
We have to use MeasureReport.group, but I'd suggest that we avoid using MeasureReport.group.polulation when we do not have any important semantics to express with it.
We'll need it when we have MeasureReports with numerators and denominators, right?
Bryn Rhodes (Apr 01 2020 at 03:32):
If you want to communicate the numerator and denominator counts, yes.
Hans Buitendijk (Apr 01 2020 at 15:05):
I updated the wiki spreadsheet with the extra attributes. @Keith Boone If you add in the example one of the two FEMA measures that are are ratio, would help clarify when the group.population in Measure is necessary, and then determine whether for a singular it is not. In the example it remains unclear what value .group adds for this measure.
Michael Donnelly (Apr 01 2020 at 15:20):
Updated Measure with subject based on @Bryn Rhodes 's feedback
{ "resourceType": "Measure", "id": "ebuqHUPORnS7mV61F6MCnFVLAo080vcLScnKBlZ5Os7A3", "name": "ICU Total Ventilators", "title": "Total Ventilators", "status": "draft", "subjectCodeableConcept": { "coding": [ { "code": "Location", "display": "Location" } ] }, "date": "2020-03-30T23:38:55Z", "publisher": "Epic", "contact": [ { "name": "Michael Donnelly", "telecom": [ { "system": "url", "value": "https://chat.fhir.org/#narrow/stream/226195-Covid-19-Response/topic/Example.20Measure.20definition", "rank": 1 }, { "system": "email", "value": "michael.donnelly@epic.com", "rank": 2 } ] } ], "description": "The number of ventilators available for use.", "useContext": [ { "code": { "system": "http://terminology.hl7.org/CodeSystem/usage-context-type", "code": "focus" }, "valueCodeableConcept": { "coding": [ { "system": "http://snomed.info/sct", "code": "840535000", "display": "COVID-19" } ], "text": "COVID-19" } } ] }
Bryn Rhodes (Apr 01 2020 at 15:49):
@Michael Donnelly , that looks good, is there a way to add url (matching the MeasureReport reference) and version?
Michael Donnelly (Apr 01 2020 at 16:24):
URL definitely. Version I'll take a look.
Michael Donnelly (Apr 01 2020 at 16:43):
Here it is with url:
{ "resourceType": "Measure", "id": "ebuqHUPORnS7mV61F6MCnFVLAo080vcLScnKBlZ5Os7A3", "url": "http://open.epic.com/Measure/ICU-Total-Ventilators-Facility", "name": "ICU Total Ventilators", "title": "Total Ventilators", "status": "draft", "subjectCodeableConcept": { "coding": [ { "code": "Location", "display": "Location" } ] }, "date": "2020-03-30T23:38:55Z", "publisher": "Epic", "contact": [ { "name": "Michael Donnelly", "telecom": [ { "system": "url", "value": "https://chat.fhir.org/#narrow/stream/226195-Covid-19-Response/topic/Example.20Measure.20definition", "rank": 1 }, { "system": "email", "value": "michael.donnelly@epic.com", "rank": 2 } ] } ], "description": "The number of ventilators available for use.", "useContext": [ { "code": { "system": "http://terminology.hl7.org/CodeSystem/usage-context-type", "code": "focus" }, "valueCodeableConcept": { "coding": [ { "system": "http://snomed.info/sct", "code": "840535000", "display": "COVID-19" } ], "text": "COVID-19" } } ] }
Michael Donnelly (Apr 01 2020 at 16:45):
I'll have to check with someone on where and how we capture version. This isn't going to be high priority - these examples won't make it into the IG; we want to use the canonical CDC ones.
Michael Donnelly (Apr 01 2020 at 16:45):
(I wanted to make these to make sure our data model had the relevant concepts and as an example for other people working on similar efforts.)
Keith Boone (Apr 01 2020 at 17:15):
It adds little value for producers, it's more like for consumers.
Bryn Rhodes (Apr 01 2020 at 17:32):
Looking great, from a semantic perspective this looks like a continuous variable measure, can you set the scoring to reflect that? And the type is a structure measure. With the subject as Location, that implies the population basis is Location (meaning the actual members of the population are expected to be Locations). Are there criteria related to selecting the locations that are included in that population? Or is it just "Locations at Organization X"?
Michael Donnelly (Apr 01 2020 at 17:43):
At this point, it's just "Locations at Organization X"
Michael Donnelly (Apr 01 2020 at 17:45):
Here are two examples with scoring specified.
Michael Donnelly (Apr 01 2020 at 17:47):
A single number, the total number of ventilators at the organization:
{ "resourceType": "Measure", "id": "ebuqHUPORnS7mV61F6MCnFVLAo080vcLScnKBlZ5Os7A3", "url": "http://open.epic.com/Measure/ICU-Total-Ventilators-Facility", "name": "ICU Total Ventilators", "title": "Total Ventilators", "status": "draft", "subjectCodeableConcept": { "coding": [ { "code": "Location", "display": "Location" } ] }, "date": "2020-03-30T23:38:55Z", "publisher": "Epic", "contact": [ { "name": "Michael Donnelly", "telecom": [ { "system": "url", "value": "https://chat.fhir.org/#narrow/stream/226195-Covid-19-Response/topic/Example.20Measure.20definition", "rank": 1 }, { "system": "email", "value": "michael.donnelly@epic.com", "rank": 2 } ] } ], "description": "The number of ventilators available for use.", "useContext": [ { "code": { "system": "http://terminology.hl7.org/CodeSystem/usage-context-type", "code": "focus" }, "valueCodeableConcept": { "coding": [ { "system": "http://snomed.info/sct", "code": "840535000", "display": "COVID-19" } ], "text": "COVID-19" } } ], "scoring": { "coding": [ { "code": "continuous-variable", "display": "Continuous Variable" } ] } }
Michael Donnelly (Apr 01 2020 at 17:48):
A rate, the percent of patients who were screened for COVID-19:
{ "resourceType": "Measure", "id": "er5hkfRvpy.uu4BZHxPJPlMySI1rbVYY96z6LzV9o.pY3", "url": "http://open.epic.com/Measure/MR-COVID-19-SCREENING-RATE-Facility", "name": "MR COVID-19 SCREENING RATE", "title": "Screening Rate", "status": "draft", "subjectCodeableConcept": { "coding": [ { "code": "Location", "display": "Location" } ] }, "date": "2020-03-30T17:03:27Z", "publisher": "Epic", "contact": [ { "name": "Michael Donnelly", "telecom": [ { "system": "url", "value": "https://chat.fhir.org/#narrow/stream/226195-Covid-19-Response/topic/Example.20Measure.20definition", "rank": 1 }, { "system": "email", "value": "michael.donnelly@epic.com", "rank": 2 } ] } ], "description": "Percentage of patients with a face-to-face, telehealth, telephone, or admission encounter for whom a COVID-19 communicable disease screening was performed.", "useContext": [ { "code": { "system": "http://terminology.hl7.org/CodeSystem/usage-context-type", "code": "focus" }, "valueCodeableConcept": { "coding": [ { "system": "http://snomed.info/sct", "code": "840535000", "display": "COVID-19" } ], "text": "COVID-19" } } ], "scoring": { "coding": [ { "code": "proportion", "display": "Proportion" } ] }, "group": [ { "population": [ { "code": { "coding": [ { "code": "numerator", "display": "Numerator" } ] }, "description": "Patients for whom a COVID-19 communicable disease screening was performed." }, { "code": { "coding": [ { "code": "denominator", "display": "Denominator" } ] }, "description": "Patients with a face-to-face, telehealth, telephone, or admission encounter." } ] } ] }
Gino Canessa (Apr 01 2020 at 18:19):
Should we be setting Measure.type to structure for things like bed counts?
Eric Haas (Apr 01 2020 at 18:25):
type = structure : A measure that focuses on a health care provider's capacity, systems, and processes to provide high-quality care.
so yes
Gino Canessa (Apr 01 2020 at 18:28):
Question becomes what codes to use in other covered Measures.. I think most should be clear: COVID-19 positive patients should be coded patient-reported-outcome, etc.
Gino Canessa (Apr 01 2020 at 18:28):
I'll take a stab at adding these to the measures I have and can review after that.
Michael Donnelly (Apr 01 2020 at 18:29):
I haven't read it, but my hunch would be that means the patients themselves reported it.
Gino Canessa (Apr 01 2020 at 18:30):
Ah, yep. copied wrong value from the table. Should be outcome
Gino Canessa (Apr 01 2020 at 18:35):
Number of patients hospitalized? Leaning towards structure, but could see either
Gino Canessa (Apr 01 2020 at 18:50):
Ok, updated the Measure Bundle here. Is the bundle alright for people to review, or are individual resource files better?
Gino Canessa (Apr 01 2020 at 19:11):
Looking at the FEMA form, there are fields that overlap with CDC and have different names. Would it make sense to add these mappings into useContext or some such? Could then search for Measure resources with a context of system-for-cdc and get back all the measures (with codes for the fields) that you need to build the report.
Thoughts?
Michael Donnelly (Apr 01 2020 at 19:55):
I'm thinking about things like that right now, Gino.
Michael Donnelly (Apr 01 2020 at 19:55):
My gut is that they'd be separate canonical Measures that represent the same data.
Michael Donnelly (Apr 01 2020 at 19:55):
I'll probably have more idea about it after a meeting I'm about to have now.
Gino Canessa (Apr 01 2020 at 20:01):
Sounds good - there are only a few fields that overlap, but it seems odd that we would generate separate measures for them (e.g., number of positive covid tests)
Bryn Rhodes (Apr 01 2020 at 21:25):
useContext is about when, where, and how it is appropriate to use the artifact. Topic is also available for things like categories, could set up a codesystem with a code for COVID-19 Response and have all the related artifacts use that for a topic.
Bryn Rhodes (Apr 01 2020 at 21:26):
The useContext in the examples is great too though, that's exactly the kind of thing I would expect to see there for focus.
Josh Mandel (Apr 01 2020 at 21:34):
Would you suggest populating topic and useContext @Bryn Rhodes ?
Bryn Rhodes (Apr 01 2020 at 21:45):
I would say yes if it's intended for a repository. It's easier for a repository to index just the topic, but the more structured useContext is useful once you have found the artifact.
Grahame Grieve (Apr 01 2020 at 23:41):
I'm mystified by these Measures. None of them actually say what they are doing anywhere. Its all off the table in secret knowledge held somewhere.
Grahame Grieve (Apr 01 2020 at 23:42):
or at least, if that's not true, I can't see how it's not true
Grahame Grieve (Apr 01 2020 at 23:43):
Also, just about everyone of them has bad terminology practices in them. I'm going to release a validator shortly that pays more attention to Measure and MeeasureReport resources
Josh Mandel (Apr 01 2020 at 23:59):
Grahame, there is indeed active discussion about how to write down computable details for the CDC measures. At least ways to say what are the filters that you might apply to a set of FHIR locations in order to determine which ones match a filter. That type of thing.
Josh Mandel (Apr 01 2020 at 23:59):
But that's only relevant once you get to automated reporting...
Grahame Grieve (Apr 02 2020 at 00:02):
well, there's kind of two parts to it :
- what does a valid MeasureReport look like
- how do you calcuate the scores
I think that the first should be specified in Measure, even if the second is not
Josh Mandel (Apr 02 2020 at 00:03):
How do you specify that in a Measure?
Josh Mandel (Apr 02 2020 at 00:03):
I think we would be grateful for any advice on this front.
Bryn Rhodes (Apr 02 2020 at 00:18):
There are MeasureReport profiles in DEQM, specifically for a summary report like this one: http://hl7.org/fhir/us/davinci-deqm/2020Feb/StructureDefinition-summary-measurereport-deqm.html
Bryn Rhodes (Apr 02 2020 at 00:19):
And the Measure profiles in the Quality Measure IG that specify elements according to scoring type of the measure: http://hl7.org/fhir/us/cqfmeasures/2020Feb/profiles.html
Josh Mandel (Apr 02 2020 at 00:30):
Okay, good -- so far we've been working on instance examples of Measure Reports; next step is defining constraints (in profiles) to match. It's a bit painful to need canonical Measure instances and MeasureReport profiles for every canonical measure.
Grahame Grieve (Apr 02 2020 at 00:32):
I too would hope that the important parts of the profile would be implied by the Measure but I'll review
Bryn Rhodes (Apr 02 2020 at 00:32):
You shouldn't need MeasureReport profiles for every Measure, the Measure content drives what a valid MeasureReport for that Measure looks like.
Gino Canessa (Apr 02 2020 at 00:37):
In the Measure resources on GH, each one should contain all the text from the CDC table. If that is insufficient description, we can add more - though I would definitely want specifics.
Grahame Grieve (Apr 02 2020 at 00:38):
so
- the possible values for score are driven by scoring type - but how can you tell what units are expected?
- There should be an entry for each population for stratum & component
- There should be a value set on the Measure.group.stratifier to tell you waht possible strata to expect
- There should only be subjectResults if type = subject-list
Grahame Grieve (Apr 02 2020 at 00:39):
?
Bryn Rhodes (Apr 02 2020 at 00:43):
Expected units is a gap, could cover with an extension like Questionnaire has: http://hl7.org/fhir/extension-questionnaire-unit.html
Michael Donnelly (Apr 02 2020 at 00:43):
Gino Canessa said:
In the
Measureresources on GH, each one should contain all the text from the CDC table. If that is insufficient description, we can add more - though I would definitely want specifics.
I'm having trouble finding these. Can you link to them?
Gino Canessa (Apr 02 2020 at 00:45):
Sure, they're all in a single bundle here. I can have the utility dump them separately if that's easier as well.
Grahame Grieve (Apr 02 2020 at 00:46):
ok, the validator will support http://hl7.org/fhir/extension-questionnaire-unit.html for that purpose
Michael Donnelly (Apr 02 2020 at 00:46):
Ooohhh, I was looking on https://github.com/AudaciousInquiry/saner-ig. That'd be why I couldn't find it. :)
Bryn Rhodes (Apr 02 2020 at 00:47):
There should be an entry for each population for stratum & component
There should be a population entry for each population specified in the Measure, and for each stratifier, for each population, yes
Grahame Grieve (Apr 02 2020 at 00:47):
And when should there be a score present?
Bryn Rhodes (Apr 02 2020 at 00:49):
There should be a value set on the Measure.group.stratifier to tell you waht possible strata to expect
This makes sense, it's been implied by the CQL, would be good to make it explicit.
Gino Canessa (Apr 02 2020 at 00:49):
Michael Donnelly said:
Ooohhh, I was looking on https://github.com/AudaciousInquiry/saner-ig. That'd be why I couldn't find it. :)
Yes, I believe Keith is still getting the build working (there is a single report sample in a branch, but that's it). I've tried to mesh all the feedback here and there together.
Bryn Rhodes (Apr 02 2020 at 00:51):
There should only be subjectResults if type = subject-list
Yes, the Summary Report profile and Individual Report profiles disallow this element
Gino Canessa (Apr 02 2020 at 00:52):
Grahame Grieve said:
And when should there be a score present?
How do you describe that in the Measure? Or is that for the profile part?
Bryn Rhodes (Apr 02 2020 at 00:52):
And when should there be a score present?
The Individual and Summary Report profiles require a score. More generally, should be present if type != data-collection
Grahame Grieve (Apr 02 2020 at 00:52):
subject-list doesn't get a score either?
Gino Canessa (Apr 02 2020 at 00:54):
Bryn, so every report we generate should have a score, even if it's the same as the count?
Bryn Rhodes (Apr 02 2020 at 01:00):
Yes, score is the measure score, count is the number of members in the population. For example, in the case of ICU Hospital Beds, the population is actually ICU Locations, so the count would be number of ICU Locations, the score is number of beds, that's the continuous variable being measured.
Bryn Rhodes (Apr 02 2020 at 01:01):
Yes, subject-list gets a score, only data-collection reports don't have a score.
Bryn Rhodes (Apr 02 2020 at 01:04):
The quality reporting topic does cover requirements on a MeasureReport based on the Measure: http://hl7.org/fhir/clinicalreasoning-quality-reporting.html#reporting-population-data
Gino Canessa (Apr 02 2020 at 01:04):
So for the numBeds Measure, the population is the number of locations (e.g., 1) and the score is the number of beds? (just making sure I have it correct).
Bryn Rhodes (Apr 02 2020 at 01:06):
Right, it's a structure measure, the subject is Location, so you're defining a Measure on the population of "Locations", and it's a continuous variable measure, where the measure observation (the function calculated for each member of the population) is "number of beds"
Bryn Rhodes (Apr 02 2020 at 01:06):
And the result is aggregated according to the "aggregate-method"
Josh Mandel (Apr 02 2020 at 01:10):
I'm just repeating to make sure I understand: if the MeasureReport.subject is a facility-level Location with 20 ICU beds, then population.count = 1 and measureScore = 20? This is an important difference from what we've understood 'til now.
Gino Canessa (Apr 02 2020 at 01:12):
So does this look correct, or need changes?
Josh Mandel (Apr 02 2020 at 01:12):
And if a health system wanted to generate a single report on 3 separate facilities each with 20 ICU beds (I'm not suggest we do this, just trying to grok the model), they'd populate 3 .groups entries, each with a count=1 and a measureScore = 20? Hmm, I guess not because it'd be unclear which facility each .group entry was talking about.
Gino Canessa (Apr 02 2020 at 01:13):
In that particular case, I'd assume each group would be coded to match the facility?
Josh Mandel (Apr 02 2020 at 01:14):
I'm not following
Josh Mandel (Apr 02 2020 at 01:16):
There's no place to list specific facility-level locations in a . group, and the entire report can only ever be able one subject. I'm struggling to understand what the population means here. (Previously when I thought it gave us a place to write bed counts, it made sense to me -- but now I'm back to confused.)
Bryn Rhodes (Apr 02 2020 at 01:16):
The structure supports both Individual and Summary reporting.
Bryn Rhodes (Apr 02 2020 at 01:17):
For Summary reporting, you don't specify an Individual (you could potentially specify a Group as the subject, but that's not necessary). At Summary level, you are interested in the overall result.
Josh Mandel (Apr 02 2020 at 01:18):
But for summary reporting, what's the point of filling out a population with count of 1, if you can't say which one?
Bryn Rhodes (Apr 02 2020 at 01:18):
These are Summary reports, so your subject-type is Location, and in the case where you're running over an Organization with different Locations, your population would be the 3 Locations, and your count would be the sum of beds in those 3 locations.
Josh Mandel (Apr 02 2020 at 01:18):
But the business requirement is to provide facility level summaries.
Josh Mandel (Apr 02 2020 at 01:19):
So that effectively means one MeasureReport per facility?
Josh Mandel (Apr 02 2020 at 01:19):
By analogy, if I'm doing a summary on a Group, would I write a population with count of 1 to mean "summary for 1 group" rather than 200 to mean "200 patients in the group"?
Bryn Rhodes (Apr 02 2020 at 01:20):
I see, you want to say for each Location, how many beds. I misunderstood that, so it is Individual Level, yes, where the Subject is a single Location.
Bryn Rhodes (Apr 02 2020 at 01:21):
The count is still required, even though it will be 0 or 1 in that case, because it indicates membership in the different criteria.
Bryn Rhodes (Apr 02 2020 at 01:22):
Doesn't tell you anything more in this case, but for example, a proportion measure, the counts tell you where the member landed in the different criteria.
Gino Canessa (Apr 02 2020 at 01:22):
Perhaps you could have a single group representing the 3 locations, with a stratifier used to separate them? Not sure how you actually link each location to each stratifier group though.
Bryn Rhodes (Apr 02 2020 at 01:24):
You could do that with stratifiers, but correct me if I'm wrong Josh, but I thought that was considered and not used because the existing metrics weren't defined that way?
Josh Mandel (Apr 02 2020 at 01:24):
The count is still required, even though it will be 0 or 1 in that case, because it indicates membership in the different criteria.
Technically optional, but I guess you're saying we should populate
Grahame Grieve (Apr 02 2020 at 01:25):
I had expected that you would have a population of beds, divided up by location, and report the count of % of what are free
Grahame Grieve (Apr 02 2020 at 01:25):
though free is a relative term here
Josh Mandel (Apr 02 2020 at 01:25):
We're definitely planning to KISS. and follow the CDC measures as closely as possible. But I want to understand how these resource are designed to work so we know when we're bending the design.
Josh Mandel (Apr 02 2020 at 01:26):
@grahame, my original perspective was to report on all beds, stratified by type, status, location, etc. Gino worked up examples like https://github.com/microsoft-healthcare-madison/learning-spike-erp/blob/master/generated/t0/Org-1065-measureReports.json
Josh Mandel (Apr 02 2020 at 01:27):
But CDC now has a set of very simple reporting requirements like "tell us how many ICU beds you have available". And we're basically trying to turn each of these into an independent Measure.
Bryn Rhodes (Apr 02 2020 at 01:32):
So it seems like the most straightforward thing there is subjectType is Location, scoring is continuous-variable, type is structure, and in the MeasureReport, the score is the number of beds.
Josh Mandel (Apr 02 2020 at 01:33):
Is this consistent with the link Gino posted above?
Josh Mandel (Apr 02 2020 at 01:33):
What does " type is structure" mean?
Josh Mandel (Apr 02 2020 at 01:33):
I'm sorry for the remedial level questions, but I'm struggling a bit here :)
Bryn Rhodes (Apr 02 2020 at 01:34):
Not at all, apologies the documentation isn't as helpful as it should be.
Bryn Rhodes (Apr 02 2020 at 01:34):
The type element on the Measure
Josh Mandel (Apr 02 2020 at 01:34):
I may not have read everything I should (just the core spec)
Bryn Rhodes (Apr 02 2020 at 01:35):
There can be process measures, outcome measures, structure measures, and a few others.
Bryn Rhodes (Apr 02 2020 at 01:35):
Structure measures are specifically about capacity questions like this (i.e. not directly patient related)
Josh Mandel (Apr 02 2020 at 01:36):
Does .type have any impact on interpretation or constraints?
Josh Mandel (Apr 02 2020 at 01:36):
Or just a kind of informative tag?
Bryn Rhodes (Apr 02 2020 at 01:36):
Just an informative tag
Bryn Rhodes (Apr 02 2020 at 01:37):
Is this consistent with the link Gino posted above?
Yes, and also with the examples Michael Donnelly has been producing.
Josh Mandel (Apr 02 2020 at 01:39):
Wait, the last MeasureReport I saw from Michael was using the population count to indicate a number of beds rather than "1".
Bryn Rhodes (Apr 02 2020 at 01:41):
Yep, sure enough. Sorry, I missed that. Yes, that should be the score.
Eric Haas (Apr 02 2020 at 02:32):
so these measures and measurereports are simple compared to the example in DEQM but the profile:
file:///Users/ehaas/Documents/FHIR/Davinci-DEQM/output/StructureDefinition-indv-measurereport-deqm.html
is still (mostly )valid...sub out location for patient. re Units the score is type Quantity in MeasureReport
Michael Donnelly (Apr 02 2020 at 02:41):
Bryn Rhodes said:
Yep, sure enough. Sorry, I missed that. Yes, that should be the score.
So what should be different about it? As far as I can follow, we've got all the numbers we need, I just need to put them all in the right places.
Bryn Rhodes (Apr 02 2020 at 03:13):
@Michael Donnelly , based on the discussion on this thread, I updated the last example you posted:
{
"resourceType": "MeasureReport",
"id": "exIpgIZ2KRCpuIrsZ1v5mpBdAwVWrJ77AAAV3tpfplf5Bvzvw41h3M5wjTXNaSq-fkrRiKloO5-wUPxcp0orOxA3",
"status": "complete",
"type": "individual",
"measure": "http://open.epic.com/Measure/ICU-Total-Ventilators-Facility",
"subject": {
"reference": "Location/eDvShAeUfxUlbCN1A6OATOQ3",
"display": "River Hills Hospital System"
},
"date": "2020-04-01",
"reporter": {
"reference": "Organization/e9OtaUAOSwwwfGDaKD43WbA3",
"display": "River Hills Hospital System"
},
"period": {
"start": "2020-03-18",
"end": "2020-03-18"
},
"group": [
{
"population": [
{
"code": {
"coding": [
{
"code": "measure-population",
"display": "Measure Population"
}
]
},
"count": 1
}
],
"measureScore": {
"value": 183
}
}
]
}
Bryn Rhodes (Apr 02 2020 at 03:14):
3 changes:
typeelement changed toindividualcountelement in the measure-population group changed to 1 (because there's only one member of the population in this case)measureScoreelement set to the Quantity { value: 183 }
Grahame Grieve (Apr 02 2020 at 03:29):
what would the measure look like?
Bryn Rhodes (Apr 02 2020 at 04:04):
{
"resourceType": "Measure",
"id": "ebuqHUPORnS7mV61F6MCnFVLAo080vcLScnKBlZ5Os7A3",
"url": "http://open.epic.com/Measure/ICU-Total-Ventilators-Facility",
"name": "ICU Total Ventilators",
"title": "Total Ventilators",
"status": "draft",
"subjectCodeableConcept": {
"coding": [
{
"code": "Location",
"display": "Location"
}
]
},
"date": "2020-03-30T23:38:55Z",
"publisher": "Epic",
"contact": [
{
"name": "Michael Donnelly",
"telecom": [
{
"system": "url",
"value": "https://chat.fhir.org/#narrow/stream/226195-Covid-19-Response/topic/Example.20Measure.20definition",
"rank": 1
},
{
"system": "email",
"value": "michael.donnelly@epic.com",
"rank": 2
}
]
}
],
"description": "The number of ventilators available for use.",
"useContext": [
{
"code": {
"system": "http://terminology.hl7.org/CodeSystem/usage-context-type",
"code": "focus"
},
"valueCodeableConcept": {
"coding": [
{
"system": "http://snomed.info/sct",
"code": "840535000",
"display": "COVID-19"
}
],
"text": "COVID-19"
}
}
],
"scoring": {
"coding": [
{
"code": "continuous-variable",
"display": "Continuous Variable"
}
]
},
"group": [
{
"population": [
{
"code": {
"coding": [
{
"system": "http://terminology.hl7.org/CodeSystem/measure-population",
"code": "measure-population"
}
]
},
"criteria": {
"description": "ICU Location"
}
},
{
"code": {
"coding": [
{
"system": "http://terminology.hl7.org/CodeSystem/measure-population",
"code": "measure-observation"
}
]
},
"criteria": {
"description": "Number of ventilators available for use",
"language": "text/cql",
"expression": "Count([Device: Ventilators])"
}
}
]
}
]
}
Bryn Rhodes (Apr 02 2020 at 04:05):
Added the group element to specify the measure-population and the measure-observation.
Bryn Rhodes (Apr 02 2020 at 04:07):
The continuous-variable measure profile requires an initial-population as well, but I wonder whether that notion makes as much sense for structure measures like this. And I'm fudging on the CQL there, working on a more complete definition.
Bryn Rhodes (Apr 02 2020 at 04:09):
Looking at the "COVID Screening Measure", that one is a proportion measure, and screening measures like that are typically expressed as patient-based proportion measures, rather than a structure measure.
Bryn Rhodes (Apr 02 2020 at 04:17):
So that might look something like this:
{
"resourceType": "Measure",
"id": "er5hkfRvpy.uu4BZHxPJPlMySI1rbVYY96z6LzV9o.pY3",
"url": "http://open.epic.com/Measure/MR-COVID-19-SCREENING-RATE",
"name": "MR COVID-19 SCREENING RATE",
"title": "Screening Rate",
"status": "draft",
"subjectCodeableConcept": {
"coding": [
{
"code": "Patient",
"display": "Patient"
}
]
},
"date": "2020-03-30T17:03:27Z",
"publisher": "Epic",
"contact": [
{
"name": "Michael Donnelly",
"telecom": [
{
"system": "url",
"value": "https://chat.fhir.org/#narrow/stream/226195-Covid-19-Response/topic/Example.20Measure.20definition",
"rank": 1
},
{
"system": "email",
"value": "michael.donnelly@epic.com",
"rank": 2
}
]
}
],
"description": "Percentage of patients with a face-to-face, telehealth, telephone, or admission encounter for whom a COVID-19 communicable disease screening was performed.",
"useContext": [
{
"code": {
"system": "http://terminology.hl7.org/CodeSystem/usage-context-type",
"code": "focus"
},
"valueCodeableConcept": {
"coding": [
{
"system": "http://snomed.info/sct",
"code": "840535000",
"display": "COVID-19"
}
],
"text": "COVID-19"
}
}
],
"library": "http://open.epic.com/Library/COVID19Screening",
"scoring": {
"coding": [
{
"code": "proportion",
"display": "Proportion"
}
]
},
"group": [
{
"population": [
{
"code": {
"coding": [
{
"code": "numerator",
"display": "Numerator"
}
]
},
"description": "Patients for whom a COVID-19 communicable disease screening was performed.",
"criteria": {
"language": "text/cql.identifier",
"expression": "Initial Population"
}
},
{
"code": {
"coding": [
{
"code": "denominator",
"display": "Denominator"
}
]
},
"description": "Patients with a face-to-face, telehealth, telephone, or admission encounter.",
"criteria": {
"language": "text/cql.identifier",
"expression": "Denominator"
}
}
]
}
]
}
Bryn Rhodes (Apr 02 2020 at 04:18):
With a Covid19Screening library:
library Covid19Screening
using FHIR version '4.0.1'
include FHIRHelpers version '4.0.1'
codesystem "Observation Category Codes": 'http://terminology.hl7.org/CodeSystem/observation-category'
valueset "Encounter Inpatient": 'http://cts.nlm.nih.gov/fhir/ValueSet/2.16.840.1.113883.3.666.5.307'
valueset "Emergency Department Visit": 'http://cts.nlm.nih.gov/fhir/ValueSet/2.16.840.1.113883.3.117.1.7.1.292'
valueset "Telephone Evaluation": 'http://cts.nlm.nih.gov/fhir/ValueSet/2.16.840.1.113883.3.464.1003.101.12.1082'
valueset "Telephone Management": 'http://cts.nlm.nih.gov/fhir/ValueSet/2.16.840.1.113883.3.464.1003.101.12.1053'
valueset "COVID-19 Screening Result": 'TBD'
code "Laboratory Observation Category": 'laboratory' from "Observation Category Codes"
parameter "Measurement Period" Interval<DateTime>
context Patient
define "Inpatient, Emergency, or Telephone Evaluation or Management Encounter":
(
[Encounter: "Encounter Inpatient"]
union [Encounter: "Emergency Department Visit"]
union [Encounter: "Telephone Evaluation"]
union [Encounter: "Telephone Management"]
) Encounter
where Encounter.status = 'finished'
and Encounter.period during "Measurement Period"
define "Initial Population":
exists ("Inpatient, Emergency, or Telephone Evaluation or Management Encounter")
define "Denominator":
true
define "COVID-19 Communicable Disease Screening":
[Observation: "COVID-19 Screening Result"] Result
where exists (Result.category C where C ~ "Laboratory Observation Category")
and Result.status in { 'final', 'amended', 'corrected' }
and Result.effective during "Measurement Period"
define "Numerator":
exists ("COVID-19 Communicable Disease Screening")
Bryn Rhodes (Apr 02 2020 at 04:21):
I chose some common value sets from the published AU measures for inpatient and telehealth encounters, but obviously those are interchangeable with whatever is being developed to determine that set. And I left a TBD on the value set for the COVID-19 Screening observations.
Grahame Grieve (Apr 02 2020 at 11:29):
@Bryn Rhodes looking at your example Measure, the validator gives the following report:
WARNING: Measure: mea-0: Name should be usable as an identifier for the module by machine processing applications such as code generation [name.matches('[A-Z]([A-Za-z0-9_]){0,254}')]
WARNING: Measure.subject.ofType(CodeableConcept).coding[0]: A code with no system has no defined meaning. A system should be provided
WARNING: Measure.useContext[0].value.ofType(CodeableConcept).coding[0]: The display "COVID-19" is not a valid display for the code {http://snomed.info/sct}840535000 - should be one of ["Antibody to 2019 novel coronavirus","Antibody to 2019-nCoV","Antibody to severe acute respiratory syndrome coronavirus 2 (substance)","Antibody to severe acute respiratory syndrome coronavirus 2","Antibody to SARS-CoV-2"]
WARNING: Measure.scoring: None of the codes provided are in the value set http://hl7.org/fhir/ValueSet/measure-scoring (http://hl7.org/fhir/ValueSet/measure-scoring, and a code should come from this value set unless it has no suitable code) (codes = null#continuous-variable)
WARNING: Measure.scoring.coding[0]: A code with no system has no defined meaning. A system should be provided
ERROR: Measure.group[0].population[0].criteria: exp-1: An expression or a reference must be provided [expression.exists() or reference.exists()]
ERROR: Measure.group[0].population[0].criteria: Profile http://hl7.org/fhir/StructureDefinition/Expression, Element 'Measure.group[0].population[0].criteria.language': minimum required = 1, but only found 0
WARNING: Measure.group[0]: Measure Groups should have at least one stratifier
Grahame Grieve (Apr 02 2020 at 11:35):
and this for the second:
ERROR: Measure.library: This property must be an Array, not a a primitive property
WARNING: Measure: mea-0: Name should be usable as an identifier for the module by machine processing applications such as code generation [name.matches('[A-Z]([A-Za-z0-9_]){0,254}')]
WARNING: Measure.subject.ofType(CodeableConcept).coding[0]: A code with no system has no defined meaning. A system should be provided
WARNING: Measure.useContext[0].value.ofType(CodeableConcept).coding[0]: The display "COVID-19" is not a valid display for the code {http://snomed.info/sct}840535000 - should be one of ["Antibody to 2019 novel coronavirus","Antibody to 2019-nCoV","Antibody to severe acute respiratory syndrome coronavirus 2 (substance)","Antibody to severe acute respiratory syndrome coronavirus 2","Antibody to SARS-CoV-2"]
WARNING: Measure.scoring: None of the codes provided are in the value set http://hl7.org/fhir/ValueSet/measure-scoring (http://hl7.org/fhir/ValueSet/measure-scoring, and a code should come from this value set unless it has no suitable code) (codes = null#proportion)
WARNING: Measure.scoring.coding[0]: A code with no system has no defined meaning. A system should be provided
WARNING: Measure.group[0].population[0].code: None of the codes provided are in the value set http://hl7.org/fhir/ValueSet/measure-population (http://hl7.org/fhir/ValueSet/measure-population, and a code should come from this value set unless it has no suitable code) (codes = null#numerator)
WARNING: Measure.group[0].population[0].code.coding[0]: A code with no system has no defined meaning. A system should be provided
WARNING: Measure.group[0].population[0].criteria.language: ValueSet http://www.rfc-editor.org/bcp/bcp13.txt not found by validator
WARNING: Measure.group[0].population[1].code: None of the codes provided are in the value set http://hl7.org/fhir/ValueSet/measure-population (http://hl7.org/fhir/ValueSet/measure-population, and a code should come from this value set unless it has no suitable code) (codes = null#denominator)
WARNING: Measure.group[0].population[1].code.coding[0]: A code with no system has no defined meaning. A system should be provided
WARNING: Measure.group[0].population[1].criteria.language: ValueSet http://www.rfc-editor.org/bcp/bcp13.txt not found by validator
WARNING: Measure.group[0]: Measure Groups should have at least one stratifier
Bryn Rhodes (Apr 02 2020 at 13:28):
First example, updated:
{ "resourceType": "Measure", "id": "ebuqHUPORnS7mV61F6MCnFVLAo080vcLScnKBlZ5Os7A3", "url": "http://open.epic.com/Measure/ICU-Total-Ventilators-Facility", "name": "ICUTotalVentilators", "title": "ICU Total Ventilators", "status": "draft", "subjectCodeableConcept": { "coding": [ { "system": "http://hl7.org/fhir/resource-types", "code": "Location", "display": "Location" } ] }, "date": "2020-03-30T23:38:55Z", "publisher": "Epic", "contact": [ { "name": "Michael Donnelly", "telecom": [ { "system": "url", "value": "https://chat.fhir.org/#narrow/stream/226195-Covid-19-Response/topic/Example.20Measure.20definition", "rank": 1 }, { "system": "email", "value": "michael.donnelly@epic.com", "rank": 2 } ] } ], "description": "The number of ventilators available for use.", "useContext": [ { "code": { "system": "http://terminology.hl7.org/CodeSystem/usage-context-type", "code": "focus" }, "valueCodeableConcept": { "coding": [ { "system": "http://snomed.info/sct", "code": "840535000", "display": "Antibody to 2019 novel coronavirus" } ], "text": "COVID-19" } } ], "scoring": { "coding": [ { "system": "http://terminology.hl7.org/CodeSystem/measure-scoring", "code": "continuous-variable", "display": "Continuous Variable" } ] }, "group": [ { "population": [ { "code": { "coding": [ { "system": "http://terminology.hl7.org/CodeSystem/measure-population", "code": "measure-population" } ] }, "description": "ICU Location" }, { "code": { "coding": [ { "system": "http://terminology.hl7.org/CodeSystem/measure-population", "code": "measure-observation" } ] }, "criteria": { "description": "Number of ventilators available for use", "language": "text/cql", "expression": "Count([Supply: Ventilators])" } } ] } ] }
Bryn Rhodes (Apr 02 2020 at 13:29):
Should take care of all of those except the "Measure groups should have at least one stratifier", stratifiers are an optional aspect, we don't have any conformance requirements currently that require the use of stratifiers. Are you thinking we should require one?
Bryn Rhodes (Apr 02 2020 at 13:35):
And the second one:
{ "resourceType": "Measure", "id": "er5hkfRvpy.uu4BZHxPJPlMySI1rbVYY96z6LzV9o.pY3", "url": "http://open.epic.com/Measure/MR-COVID-19-SCREENING-RATE", "name": "MR_COVID_19_SCREENING_RATE", "title": "COVID-19 Screening Rate", "status": "draft", "subjectCodeableConcept": { "coding": [ { "system": "http://hl7.org/fhir/resource-types", "code": "Patient", "display": "Patient" } ] }, "date": "2020-03-30T17:03:27Z", "publisher": "Epic", "contact": [ { "name": "Michael Donnelly", "telecom": [ { "system": "url", "value": "https://chat.fhir.org/#narrow/stream/226195-Covid-19-Response/topic/Example.20Measure.20definition", "rank": 1 }, { "system": "email", "value": "michael.donnelly@epic.com", "rank": 2 } ] } ], "description": "Percentage of patients with a face-to-face, telehealth, telephone, or admission encounter for whom a COVID-19 communicable disease screening was performed.", "useContext": [ { "code": { "system": "http://terminology.hl7.org/CodeSystem/usage-context-type", "code": "focus" }, "valueCodeableConcept": { "coding": [ { "system": "http://snomed.info/sct", "code": "840535000", "display": "Antibody to 2019 novel coronavirus" } ], "text": "COVID-19" } } ], "library": [ "http://open.epic.com/Library/COVID19Screening" ], "scoring": { "coding": [ { "system": "http://terminology.hl7.org/CodeSystem/measure-scoring", "code": "proportion", "display": "Proportion" } ] }, "group": [ { "population": [ { "code": { "coding": [ { "system": "http://terminology.hl7.org/CodeSystem/measure-population", "code": "numerator", "display": "Numerator" } ] }, "description": "Patients for whom a COVID-19 communicable disease screening was performed.", "criteria": { "language": "text/cql.identifier", "expression": "Initial Population" } }, { "code": { "coding": [ { "system": "http://terminology.hl7.org/CodeSystem/measure-population", "code": "denominator", "display": "Denominator" } ] }, "description": "Patients with a face-to-face, telehealth, telephone, or admission encounter.", "criteria": { "language": "text/cql.identifier", "expression": "Denominator" } } ] } ] }
Michael Donnelly (Apr 02 2020 at 13:37):
Bryn Rhodes said:
Should take care of all of those except the "Measure groups should have at least one stratifier", stratifiers are an optional aspect, we don't have any conformance requirements currently that require the use of stratifiers. Are you thinking we should require one?
I don't believe so.
Hans Buitendijk (Apr 02 2020 at 14:00):
@Keith Boone , @Michael Donnelly : I created a separate Google Spreadsheet and reformatted based on discussion with Keith. Populated it for the attributes used in various examples shared so far, while limiting to the attributes to clearly define the measures (additional attributes are necessary but can be defined by the measure tool smiths). Questions: Keith - is this in-line what we talked about. Michael - anything you see missing before I start to take our respective other measures from the wiki and put it in this format as well?
https://docs.google.com/spreadsheets/d/1TS_r8gfjZPXREOvEJaAzK-rweh8whrSH9OM6ZTA4yss/edit#gid=0
Michael Donnelly (Apr 02 2020 at 14:07):
I'd make the status be draft for now (since it is)
Michael Donnelly (Apr 02 2020 at 14:08):
Do we want to include a version? I believe @Bryn Rhodes thinks it's a good idea.
Michael Donnelly (Apr 02 2020 at 14:10):
Is subject supposed to be the name of a resource type? If so, it'd just be "Location" not "BedLocation"
Hans Buitendijk (Apr 02 2020 at 14:10):
For those already issued/published to collect, it is active from that perspective. How can we distinguish that the measure is "active" from it first appearing in FHIR, which is merely a format of publication, not necessarily the effective date? Seems two different aspects. Agreed on version.
Hans Buitendijk (Apr 02 2020 at 14:11):
On subject - will fix that.
Michael Donnelly (Apr 02 2020 at 14:11):
date should be in ISO format.
Michael Donnelly (Apr 02 2020 at 14:12):
The relatedArtifact links are great - I didn't think of that.
Michael Donnelly (Apr 02 2020 at 14:12):
What are you thinking for the url? They're canonical, so maybe http://cdc.gov/Measure/numTotBeds?
Michael Donnelly (Apr 02 2020 at 14:13):
(etc)
Hans Buitendijk (Apr 02 2020 at 14:13):
Date - for this purpose, review/etc., this format is easier to read and can easily be converted when used to populate FHIR resources.
Michael Donnelly (Apr 02 2020 at 14:13):
Okay
Hans Buitendijk (Apr 02 2020 at 14:15):
for url it would be I think where it is published. So for these ones the HL7 SANER location until CDC/whomever formally publishes with a link, for yours/others the link you have once you/others make theirs available. I was going to check your example to make sure it was url not identifier.
Bryn Rhodes (Apr 02 2020 at 14:17):
@Hans Buitendijk , can we add type and scoring to that sheet?
Hans Buitendijk (Apr 02 2020 at 14:17):
The population(initialPopulation) row is meant to point to the base measure that this one is a stratification of.
Hans Buitendijk (Apr 02 2020 at 14:17):
Will do.
Hans Buitendijk (Apr 02 2020 at 14:20):
Did we agree these are type=structure for Location measures, outcome for Tests (I need to change those to ServiceRequest and Observation per above), while Patient measures are not clear. Deaths seems to be outcome, but others not necessarily.
Bryn Rhodes (Apr 02 2020 at 14:21):
And to make it explicit, observation.description? continuous-variable measures would have an observation.description, proportion measures would have numerator/denominator
Bryn Rhodes (Apr 02 2020 at 14:21):
The screening measures we have typically called process measures.
Hans Buitendijk (Apr 02 2020 at 14:22):
@Bryn Rhodes Do you want to update the type and scoring rows?
Hans Buitendijk (Apr 02 2020 at 14:25):
subject for column S, rejected, seems to be Specimen. Any disagreement?
Keith Boone (Apr 02 2020 at 14:28):
Gino Canessa said:
Question becomes what codes to use in other covered Measures.. I think most should be clear: COVID-19 positive patients should be coded
patient-reported-outcome, etc.
Outcome, not Patient-Reported-Outcome. The former is correct, the latter means something else.
Bryn Rhodes (Apr 02 2020 at 14:29):
@Hans Buitendijk , happy to
Keith Boone (Apr 02 2020 at 14:31):
Michael Donnelly said:
Bryn Rhodes said:
Should take care of all of those except the "Measure groups should have at least one stratifier", stratifiers are an optional aspect, we don't have any conformance requirements currently that require the use of stratifiers. Are you thinking we should require one?
I don't believe so.
If you think about some of the measures, they are actually strata about a larger population. In the context of reporting, the CDC report is ONE measure with multiple strata, and the FEMA report is another measure, also with multiple strata. At present we are defining them as individual measures, but I believe we need to coalesce them into a smaller group of measures. Think about the report as a unit that gets sent or reported on, and it will help us decide how to group these things.
Keith Boone (Apr 02 2020 at 14:33):
Michael Donnelly said:
What are you thinking for the
url? They're canonical, so maybehttp://cdc.gov/Measure/numTotBeds?
Canonical URLs should resolve. I'd suggest for as long as we are creating and maintaining the canonical resources, they resolve to the guide content for them. That way we don't have to manage the issues around getting CDC and FEMA to host pages ..., which they will spend absolutely 0 time on.
Bryn Rhodes (Apr 02 2020 at 14:34):
Yeah, I'm looking through this spreadsheet and I agree, all the structure measures use numTotBeds as the initial population, then cut that different ways with the measure population. Same for the patient ones.
Bryn Rhodes (Apr 02 2020 at 14:35):
Agreed on the canonical URLs, we've been using fhir.org/guides/cdc for CDC content, in anticipation of hosting it on the fhir.org registry, but not sure where this will end up being published.
Keith Boone (Apr 02 2020 at 14:37):
Until we get to publishing, let's use the values that we know will resolve. It will help implementors who want to dereference the resource.
Bryn Rhodes (Apr 02 2020 at 14:38):
Agreed
Bryn Rhodes (Apr 02 2020 at 14:38):
So, for the structure measures it's clear that the basis is Location, but for the Patient measures, subject should be Patient, yes?
Bryn Rhodes (Apr 02 2020 at 14:40):
For the time being, keep them as separate Measures, but I do think there's a way to pull them in to a few Measures with multiple groups.
Bryn Rhodes (Apr 02 2020 at 14:41):
I think the population attributes should be:
initial-population.description
measure-population.description
measure-observation.description
denominator.description
numerator.description
Bryn Rhodes (Apr 02 2020 at 14:41):
Okay if I make that change to the spreadsheet?
Michael Donnelly (Apr 02 2020 at 14:43):
If we're using urls that we know will not be the permanent ones, I want to make a plug again for status to be draft.
Keith Boone (Apr 02 2020 at 14:45):
Measures on Orders are Process measures, of utilization/capacity are structure measures, test results and deaths are outcome measures, according to the standard definitions. Self (Patient) reported data would be Patient-reported outcomes. Measures that include multiple types (e.g., FEMA which includes orders, results) are composite.
Bryn Rhodes (Apr 02 2020 at 14:59):
I set type and scoring for all the measures to what I think are the right values. Please review and let me know if I've picked the wrong type for any of them.
Bryn Rhodes (Apr 02 2020 at 15:03):
I see that the subject for the numVent measures is DeviceDefinition, which raises some questions for me. 1. DeviceDefinition is a definitional resource, so I wouldn't expect it to be used to capture actual Devices, yes? 2. It seems inconsistent to say that the structure measures for ICU beds are Location, but the structure measures for ventilators are Device-based? Seems like the ventilators one should also be Location?
Josh Mandel (Apr 02 2020 at 15:08):
If I'm reading your last question correctly Bryn, You are basically asking us to think about reporting the number of beds available in a facility and the level of ventilators available in a facility. For both of these would be talking about the facility as a location.
Josh Mandel (Apr 02 2020 at 15:09):
(it's an incidental and confusing detail, relative to this reporting aim, that beds themselves can be as locations in FHIR.)
Bryn Rhodes (Apr 02 2020 at 15:15):
Can they also be seen as Devices?
Hans Buitendijk (Apr 02 2020 at 15:15):
@*Keith Boone said in stream:Covid-19+Response topic:Example+Measure+definition near:19267051 that we should point the url to HL7 SANER IG. Agreed for those, but for non-CDC/FEMA we are going to add here, need to point to their location when available.
BTW, not sure why I don't seem to get the reference as a hyperlink. Is that another one of those markdown syntax things rather than a wysiwig/etc.
Hans Buitendijk (Apr 02 2020 at 15:17):
If we think of beds as the beds themselves and count them, probably DeviceDefinition. If you want to reference the actual bed, not a count, then Device. Here I understand the count to be of locations where a bed can be located for care.
Hans Buitendijk (Apr 02 2020 at 15:20):
Or is the intent to state that there is a population of actual beds and ventilators (Device), and you count that? I'll fix the ventilator measures to use Device as subject, not DeviceDefinition. Concern though when we get to PPEs . That would be more count of kinds of.
Josh Mandel (Apr 02 2020 at 15:24):
Bryn Rhodes said:
Can they also be seen as Devices?
Beds? Yep. It's just that Location directly captures a bunch of details like operational status, acuity, etc.
Josh Mandel (Apr 02 2020 at 15:26):
Keith Boone said:
In the context of reporting, the CDC report is ONE measure with multiple strata, and the FEMA report is another measure, also with multiple strata.
That doesn't sound right to me. CDC "ICU BEDS" and "ICU BED OCCUPANCY" can be two strata of one measure, for sure; but "DEATHS" is not part of that measure (or at least I can't see how).
Josh Mandel (Apr 02 2020 at 15:28):
In any case, I thought we discussed this on Tuesday's call and decided that even though we see the connections to tie these measures together, we were still going to proceed as though they were independent, to reduce complexity and initial effort. Just to be clear: do we want to re-visit this decision (in light of having reconsidered, or better understood the problem, or widened the consensus group)?
Gino Canessa (Apr 02 2020 at 15:53):
Michael, I don't see type in the last examples you posted
Bryn Rhodes (Apr 02 2020 at 16:02):
So, there are two aspects to defining the population criteria, and they are about making it as easy as possible to express the criteria. There is the _subject_, which characterizes the context in which the criteria are expressed, it's basically the Compartment in which the criteria are to be interpreted. And there is the _basis_, which determines what kinds of things are actually in the population. The _scoring_ then layers on top of that to decide whether what we're measuring is a proportion or a continuous variable, or a ratio, or whatever. And obviously, a proportion measure can be expressed as two continuous variable measures that you would then have to understand as the numerator and denominator of a fraction. After looking at this spreadsheet, that is exactly what is being done with a lot of these continuous-variable measures, they are expressing the numerator and denominator of a proportion, but it's relevant because we actually care what the values are, not just the resulting fraction.
Bryn Rhodes (Apr 02 2020 at 16:07):
So in the case of beds vs ventilators, if the subject is Location, then the criteria are expressed in terms of a Location, and when an author says that the measure observation is "Count(Device)", it's counting the set of Devices _in the Location_.
Josh Mandel (Apr 02 2020 at 16:41):
Adding CQL in here as a requirement or as the way to capture logic scares me a bit; for simple constraints on, say, bed type, is this really the preferred approach?
Bryn Rhodes (Apr 02 2020 at 16:42):
So looking across all the measures, the function is always Count() (or some proportion of those resulting counts for the few proportion measures), so it would potentially be even simpler if we thought of these as cohort measures. Cohort measure scoring is just the count of members in the population, so for the Location structure measures, subject is Location, basis is Device, so the criteria is just [Device: "ICU Bed"] or [Device: "Ventilator"]
Bryn Rhodes (Apr 02 2020 at 16:43):
To be clear, I'm not suggesting it as a requirement, it helps me think through the actual computational requirements though. And it does provide a way to communicate those requirements, even if it's not used to do the actual computation.
Abbie Watson (Apr 02 2020 at 16:44):
Josh Mandel said:
Adding CQL in here as a requirement or as the way to capture logic scares me a bit; for simple constraints on, say, bed type, is this really the preferred approach?
Agreed. As much as there are some CQL advocates out there, it's not universally used and is in the realm of jargon and wonkiness.
Josh Mandel (Apr 02 2020 at 16:44):
it does provide a way to communicate those requirements
@Bryn Rhodes my concern is that's a way that's theoretically computable, but doesn't help people in practice.
Bryn Rhodes (Apr 02 2020 at 16:52):
That's fair, but it's tangential to my main question there, which is would it be simpler to treat these as cohort measures?
Bryn Rhodes (Apr 02 2020 at 16:53):
My sense is that it would, because then we're always just counting, and it's clear that that's what we're doing. We don't have to state count anywhere, just define the criteria (in narrative) for the members of the populations we're counting.
Gino Canessa (Apr 02 2020 at 17:02):
I feel like the cohort measure makes more sense (and is closer to what we started with). So, the counts would then be the counts, and the score would be: a) the same number b) something else c) not present?
Grahame Grieve (Apr 02 2020 at 20:04):
we don't have any conformance requirements currently that require the use of stratifiers. Are you thinking we should require one?
no. I've taken that out. I don't know what I was thinking
Grahame Grieve (Apr 02 2020 at 20:07):
@Bryn Rhodes still one error on your first example:
ERROR: Measure.group[0].population[0]: Profile http://hl7.org/fhir/StructureDefinition/Measure, Element 'Measure.group[0].population[0].criteria': minimum required = 1, but only found 0
Gino Canessa (Apr 02 2020 at 20:20):
Posted a new set for feedback. These use cohort scoring so that the counts are the things being counted. I also updated all the measure criteria fields to use the definition provided (e.g., the CDC field numTotBeds has a text definition).
The repo has all the current CDC fields, will get started on the FEMA ones later. If there's anything missing/incorrect/etc., please let me know.
Keith Boone (Apr 02 2020 at 21:30):
@Josh Mandel I'm saying that the CDC Report is ONE Measure, with multiple strata. Yes, it could be modeled as a collection of individual reports, but CDC isn't asking you to send multiple reports, it's asking for ONE report with multiple values that are strata.
Keith Boone (Apr 02 2020 at 21:31):
When a hosptial administrator says "CDC Report", they are talking about ALL the measures in it. That's an audience that we need to think about when we model. Yes, in the background they are collections of smaller things. But the canonical CDC report should probably be what they are asking be measured in a report.
Gino Canessa (Apr 02 2020 at 21:39):
Keith, are you saying that we should try to model everything in a single Measure? I'm not sure the semantics allow for counting beds, vents, patients, etc. in just one.
Bryn Rhodes (Apr 02 2020 at 21:40):
Right, would need to be a Measure per kind of thing. One for beds, one for vents, one for patients.
Keith Boone (Apr 02 2020 at 21:43):
The focus should be the 840539006 Disease caused by severe acute respiratory syndrome coronavirus 2 (disorder)
Not the antibody.
Gino Canessa (Apr 02 2020 at 21:45):
I worry about the complexity vs gain. In beds, CDC is asking for: total, total occupied, total ICU, total ICU occupied. In patients, they are asking for counts on: hospitalized, ventilated, onset under care, overflow, overflow ventilated, and died.
Then, to actually build the report you need to grab the reports for Beds, Patients, and Ventilators and reconstruct it by parsing out the groups across them.
Additionally, if (when?) CDC adds a new field, we'll have to change the canonical instead of adding another measure.
Keith Boone (Apr 02 2020 at 21:45):
Gino, you may be right about that for beds/vents/patients. But where the "Report" being requested by an Agency is focused on the same kind of thing, they should be in the same report.
Keith Boone (Apr 02 2020 at 21:46):
If CDC mandates a different report, we SHOULD change the canonical for it. It's no longer the current report they are asking for.
Keith Boone (Apr 02 2020 at 21:47):
Or at least the version...
Gino Canessa (Apr 02 2020 at 21:47):
We need to change the IG - if they are just adding a new field isn't it preferable to leave all the existing reports in place and just add the new one?
Edit: thinking in the context of systems that need to actually get the data and fill it out
Keith Boone (Apr 02 2020 at 21:48):
I'm expecting at SOME stage we will wind up with a Measure Registry, NOT an IG. The IG is to get things moving, and will serve as a registry for the time being.
Keith Boone (Apr 02 2020 at 21:50):
There's two parts of this, as you point out @Gino, what consumers do, and what producers have to do.
Gino Canessa (Apr 02 2020 at 21:50):
I guess where I'm stuck is that having a MeasureReport split out by each of the classifications is more complicated, and we still need to fetch multiple MeasureReport resources in order to construct a single CDC report. We're adding complexity, so what do we gain?
Keith Boone (Apr 02 2020 at 21:53):
When two "Measures" share the same initial population, but are talking about orthogonal (or hierarchical/orthogonal) subsets, I think they should be part of same measure (with strata). When they have different potentially overlapping strata, they should be in different Measures (we can only go so far combining strata).
Keith Boone (Apr 02 2020 at 21:53):
You'll recall that I started with Group which doesn't quite have that problem ...
Keith Boone (Apr 02 2020 at 21:54):
And that measure is defined in terms of Group ...
and so there are likely some fairly common groups
Gino Canessa (Apr 02 2020 at 21:58):
Trying to make sure I understand what you are proposing. For beds, CDC wants: total, total inpatient, total inpatient occupied, total icu, total icu occupied.
How many measures do you break that into, and on which lines?
Keith Boone (Apr 02 2020 at 22:02):
One measure, where initial-population = total for each, with groups in the measure, one each for total, total-inpatient, total-inpatient-occupied, total-icu, total-icu-occupied where group contains the count in measure-population.
Because these are all about beds that are a subset of total.
Hans Buitendijk (Apr 02 2020 at 22:03):
Also need to consider that CDC is o.k. with back-reporting of whatever one did not have before. Not as likely when automated, but depends on how folks would start to use it.
Gino Canessa (Apr 02 2020 at 22:07):
So with this model we get
- Bed report: 5 groups
- Patient report: 5/6 groups (not sure if living and dead patients can be mixed here)
- Ventilator report: 2 groups
To generate the CDC report, you need to pull those 3 (or 4) reports and get the field values from each group.
Is that correct?
Bryn Rhodes (Apr 02 2020 at 22:08):
Yes, that sounds right to me. And that is the whole reason the Measure is structured that way, so you can report different aspects of the same initial population.
Keith Boone (Apr 02 2020 at 22:15):
patients who have died on a given day are a subset of those who were alive on that day. But the stratum is based on the event of deaths.
The ugly part: Patient who has been waiting on a bed in an overflow area for days on a makeshift ventilator who dies on day 16. I hope NOBODY ever has to deal with counting that person in a report. But which of the strata do they fit into?
Gino Canessa (Apr 02 2020 at 22:15):
Great! I understand both models, but I'm not sure I have a strong opinion either way.
Individual fields feels simpler, but grouped feels cleaner. I can try to export sets for both before tomorrow's meeting if it's something we want to look at on the call.
Keith Boone (Apr 02 2020 at 22:16):
Yes please.
Gino Canessa (Apr 02 2020 at 22:16):
The field info for 'died' specifically includes people waiting on capacity (beds/vents).
Gino Canessa (Apr 02 2020 at 22:18):
Back online in a few, dinner time
Keith Boone (Apr 02 2020 at 22:18):
Right... so they definitely fit into that category. But they also consumed overflow space on that day, even if they are no longer at the end of the day. And they consumed a ventilator (even if makeshift) for some part of that day.
Keith Boone (Apr 02 2020 at 22:19):
More clarifications for @Austin Kreisler to track down with NHSN
Michael Donnelly (Apr 03 2020 at 15:11):
Oh man, I've been way off on this. I thought a Measure was a single metric not a whole set of them.
Gino Canessa (Apr 03 2020 at 15:58):
ok, just finished:
- Bundle with individual CDC measures
- Bundle with CDC beds as a single Measure
- Bundle with individual measure reports
- Bundle with single bed measure report
Bryn Rhodes (Apr 03 2020 at 16:17):
Those look great, one thing is that for a cohort measure the group is "initial-population"; "measure-population" is only appropriate for continuous-variable measures.
Bryn Rhodes (Apr 03 2020 at 16:20):
And I've been looking at the definitions for the CDC measures and unless I'm reading it wrong, the patient ones aren't all about the same set of initial patients. Specifically, there are two groups, patients in inpatient beds, and patients in other locations, and then the numC19Died metric is about both of those groups. So I think modeling this as cohort measures across the board is the simplest way to go, then it's just a matter of packaging whether those are all in the same measure resource or not.
Bryn Rhodes (Apr 03 2020 at 16:21):
And on the single Measure, Gino, each of those cohort initial populations would be a separate group.
Bryn Rhodes (Apr 03 2020 at 16:24):
I'd also like to add a "basis" row to make clear what the kinds of things are in the populations being counted. Okay if I add that row to the spreadsheet?
Hans Buitendijk (Apr 03 2020 at 17:00):
@Keith Boone , @Gino Canessa , @Michael Donnelly I missed conclusion on Measure.status. how to distinguish between published by CDC/FEMA (already active) vs. FHIR resource published. We need the first, but looks like in the spreadsheet Keith shared on the call it was set to "draft"
Josh Mandel (Apr 03 2020 at 17:01):
@Bryn Rhodes when working with stratifiers across multiple components (e.g., gender + age bucket, like "males age 20-30"), how does stratum.value vs stratum.componen.value work? Is stratum.value supposed to be omitted? The example here uses male for both, and I'm not sure what it means:
Bryn Rhodes (Apr 03 2020 at 17:25):
Yes, they are exclusive, you either have a stratum.value or you have a set of stratum.component.value. If you are stratifying by multiple criteria, the components let you state the codes independently, rather than having to mash them together in a single value. The hiv-indicators examples in the spec illustrate the difference:
Measure:
image.png
Bryn Rhodes (Apr 03 2020 at 17:26):
Wow, those images didn't come through very well. Sorry, trying to provide a focus on the relevant part of the examples.
Eric Haas (Apr 03 2020 at 18:56):
@Bryn Rhodes @Gino Canessa here was my take on it in one measure with three groups beds, vents and patients .... https://healthedata1.github.io/tempIGViewer/Measure-cdc-covid.html
Gino Canessa (Apr 03 2020 at 18:58):
I don't recall having a discussion beyond noting that we currently have everything set to draft since we're still working on them.
Josh Mandel (Apr 03 2020 at 20:09):
Yes, they are exclusive, you either have a stratum.value or you have a set of stratum.component.value.
Thanks @Bryn Rhodes! Does this mean there can/could/should be a constraint to this effect?
Josh Mandel (Apr 03 2020 at 20:10):
here was my take on it in one measure with three groups bed
I don't know enough to understand @Eric Haas if your example is following the intended use of .population, or playing fast and loose with the semantics.
Gino Canessa (Apr 03 2020 at 20:38):
@Eric Haas thanks, that is close to what I came up with as well.
Right now, I am confused while turning this into a MeasureReport based on the Measure.
Specifically:
- Are we supposed to have one
stratifierwith severalstratum, or severalstratifierrecords, with onstratumeach? - Are we supposed to fill out the
populationon eachstratum?
Josh Mandel (Apr 03 2020 at 20:41):
Are we supposed to have one stratifier with several stratum, or several stratifier records, with on stratum each?
I think individual stratum elements ("strata!!" the singular is driving me crazy ;-)) are supposed to be grouped under the same stratifier if they share their set of stratifier.code values. In other words, if you want to report on
- males age 18-40
- females age 18-40
- males age 41-65
- females age 41-65
Then your stratum.code array would be [sex, age]. Since all four of these buckets are about [sex, age], the should be under the same stratifier.
Josh Mandel (Apr 03 2020 at 20:42):
Are we supposed to fill out the population on each stratum?
I also would like to know. @Bryn Rhodes ??
Bryn Rhodes (Apr 03 2020 at 20:52):
Yes, the intent being that each stratum reports the population counts for each criteria defined in the measure
Gino Canessa (Apr 03 2020 at 20:53):
Yes, that's the problem. Since we are grouping all the (e.g.) beds together, the stratifier is essentially 'counts CDC wants about beds'.
By that, the stratum.code is the field name, and there's just a single stratifier, correct?
Bryn Rhodes (Apr 03 2020 at 20:53):
Thanks @Bryn Rhodes! Does this mean there can/could/should be a constraint to this effect?
I've added this as a comment to this tracker: https://jira.hl7.org/browse/FHIR-26681
Gino Canessa (Apr 03 2020 at 20:56):
Otherwise, we can go and figure out translations for each of the fields and build up the models, but filling them out is already getting annoyingly complex.
Bryn Rhodes (Apr 03 2020 at 20:57):
Yeah, I actually don't think we need stratifiers here, we're representing all these as separate cohorts, so the reporting would just be the different cohort groups.
Bryn Rhodes (Apr 03 2020 at 20:58):
Whether that's in a single Measure/MeasureReport or multiple Measure/MeasureReports doesn't really matter.
Bryn Rhodes (Apr 03 2020 at 20:59):
Even if I were to represent these as proportion measures with initial/numerator/denominator, I still wouldn't be stratifying. Stratifiers are for things like age and gender where you're not fundamentally changing the meaning of the thing being measured, just dicing it up by different axes.
Josh Mandel (Apr 03 2020 at 21:04):
When you talk about separate cohorts, can you explain the difference between using multiple groups each with a population, or one group with multiple populations? (I think I actually have an understanding of this but I would like to confirm it :-))
Bryn Rhodes (Apr 03 2020 at 21:06):
Each group represents a different measure, so each group can only have one entry for the appropriate criteria types based on the scoring. So a proportion measure can have multiple groups, but each group only has one initial/denominator/numerator, etc.
Gino Canessa (Apr 03 2020 at 21:06):
So instead of one group for beds with multiple stratifiers, we should just promote everything to group level? (making sure I have it correct)
Bryn Rhodes (Apr 03 2020 at 21:06):
A measure can have any number of groups, but they all have to be of the same scoring type (and for the same subject and basis).
Bryn Rhodes (Apr 03 2020 at 21:07):
@Gino Canessa , yes, each of the measures have their own group. And then whether it's in one or more Measure resources is just a matter of packaging.
Josh Mandel (Apr 03 2020 at 21:08):
Each group represents a different measure,
So a Measure (with multiple .groups) can define multiple measures!
Bryn Rhodes (Apr 03 2020 at 21:09):
Yes :). And if you look at the annual update measures CMS puts out, or what NCQA delivers for HEDIS, that's fairly common. One "Measure" with multiple performance rates.
Gino Canessa (Apr 03 2020 at 21:11):
Thanks a bunch Bryn!
Bryn Rhodes (Apr 03 2020 at 21:19):
No worries, happy to help!
Gino Canessa (Apr 03 2020 at 21:39):
Pushed out an update:
- Bundle with a single Measure for all CDC fields, grouped
- Bunble with a single MeasureReport matching the above Measure
I believe these incorporate all the feedback I've received so far. If there's anything missing/wrong/could-be-better, please let me know.
I'm going to finish getting the FEMA versions up and running next.
Josh Mandel (Apr 03 2020 at 21:55):
nice -- this feels pretty OK to me! One thing we might still want to figure out how to put in-band is the "computable definition" for the various groups. CQL seems plausible but not widely understood; could also do something "funny" and just express like a search string (e.g., criteria.expression=?operational-status=U&type=ICU)
Bryn Rhodes (Apr 03 2020 at 22:04):
I have started on some of the CQL, planning on finishing that up this weekend so it's available, but totally understand if it's just informative content.
Gino Canessa (Apr 03 2020 at 22:04):
Yeah, it would be nice if multiple Expressions were allowed, so that we could have the equivalent definitions.
- we could add an extension to group/population/etc.
- we could add mappings in the
Measure.definitionelement, though they wouldn't be computable
Bryn Rhodes (Apr 03 2020 at 22:05):
Need to specify the basis though, am I okay to add that row after subject?
Bryn Rhodes (Apr 03 2020 at 22:05):
In the spreadsheet?
Bryn Rhodes (Apr 03 2020 at 22:05):
I don't want to break any tooling, so I'm waiting for confirmation before I change anything there.
Gino Canessa (Apr 03 2020 at 22:07):
I'm not using anything from there as computable, but I think Keith was. May be worth pinging him.
Gino Canessa (Apr 03 2020 at 22:08):
Oh, going back to the definitions. We could also misuse RelatedArtifact and stick them in there
Gino Canessa (Apr 03 2020 at 22:08):
Or not misuse it and create an attachment with the definitions in it.
Hans Buitendijk (Apr 03 2020 at 22:09):
Who can address the open question how to distinguish two statuses: CDC/FEMA published, thus active; FHIR resource not final, thus draft. We need to have the first one, but if Measure.status is only supposed to carry the second, we are missing an important attribute.
Bryn Rhodes (Apr 03 2020 at 22:12):
Seems like it would be an extension on the relatedArtifact referencing the source. Extension for publicationStatus?
Eric Haas (Apr 03 2020 at 22:19):
@Bryn I was confused by cohorts and started with the group as beds, vents and patients..and pops and strats as the starting point but after looking at trying to use my Measure to make the MR and reading this there are only groups and pops no strats? right
Hans Buitendijk (Apr 03 2020 at 22:40):
@Bryn Rhodes that would work. I'll add a row under related artifact for a publication status.
Hans Buitendijk (Apr 03 2020 at 22:40):
@Keith Boone : you ready for me to make updates to the Google Sheet (jurisdiction codes, publication status, and possibly some other updates)?
Keith Boone (Apr 03 2020 at 22:43):
Not yet, spend the entire day on the phone instead of working (that's what I get to do tonight).
Hans Buitendijk (Apr 03 2020 at 22:43):
Just ping when ready. I'll update tomorrow.
Bryn Rhodes (Apr 03 2020 at 22:44):
@Eric Haas , yes, that's correct, just groups with initial-population, no stratifiers.
Gino Canessa (Apr 04 2020 at 20:00):
Added first pass of:
Eric Haas (Apr 04 2020 at 21:43):
@Gino Canessa lookin at the FEMA Measures, your scoring is a Cohort. Cohort works for just counts but not ratios which is a Proportion. Based on the QM guidance you can't have Numerator and Denominator with Type = Cohort.
... so looking at the FEMA form there are two types of scoring so 2 measures...
measure 1 scoring = Cohort: IPP only
- New Diagnostic Tests Ordered/Received
- Cumulative Diagnostic Tests Ordered/Recieved
- Cumulative Specimens Rejected*
measure 2 scoring = Proportion: IPP = Denom and Numerator
- Percent Positive among Newly Resulted Tests
- New Tests Resulted
- New Positive COVID-19 Tests
- Cumulative Percent Positive among Resulted Tests
- Cumulative Tests Performed
- Cumulative Positive COVID-19 Tests
@Bryn Rhodes can you confirm?
Eric Haas (Apr 04 2020 at 21:47):
Eric Haas: here are my takes on the CDC form for measure and measurereport : https://healthedata1.github.io/tempIGViewer/all-examples.html
Eric Haas: also a bundle for several measurereports over several days
Eric Haas: here is a Jupyter notebook converting the csv to MR and back again...
Eric Haas: :notebook: https://mybinder.org/v2/gh/Healthedata1/MyBinder/master
- click on the pyFHIR_models folder
- click on the notebook file C19_MeasureReport-CSV1.ipynb
- static view in the nbviewer available pretty quickly
- interactive view takes a while to load.
Eric Haas: note how am using the id to link the Measure to MR. Bryn talked to me about this... but afterwards I think I was being too cute by grouping them and then having multiple populations.. Think could just flatten to one count/group. and use the group codes instead of the id to link them.
Eric Haas: the measure url is all wonky btw
Hans Buitendijk (Apr 05 2020 at 00:16):
Added publicationStatus and publicationDate to the relatedArtifact in the Google spreadsheet. Sugges they are both extensions.
Bryn Rhodes (Apr 05 2020 at 01:18):
Added a tracker to consider adding to the RelatedArtifact type in R5, and add extensions for them in the QM IG for R4 usage.
Keith Boone (Apr 05 2020 at 03:42):
On cohort vs. continuous-variable, see https://www.hl7.org/fhir/clinicalreasoning-quality-reporting.html#specifying-population-criteria
Keith Boone (Apr 05 2020 at 03:45):
I think we can use relatedArtifact.document.createdDate for publicationDate
Hans Buitendijk (Apr 05 2020 at 12:38):
I'll update RelatedArtifact in the spreadsheet to use document.creation and the document.url would then better suit the link rather than relatedArtifact.url. One place to shop. Still keep the status on relatedArtifact.
Gino Canessa (Apr 06 2020 at 14:17):
Good catch @Eric Haas , thanks. I think I'll propose removing the ratios from the Measure/MeasureReport, making sure that the numerator and denominator are there individually. Anyone making the spreadsheet can do the division themselves.
Keith Boone (Apr 06 2020 at 15:44):
Gino, I think I disagree. Yes, you can do the division, but there's potentially also addition and subtraction in measures (inclusions and exclusions). Better to leave the computing a done thing, instead of an expected thing. Some consumers aren't going to want to compute these, just use them.
Keith Boone (Apr 06 2020 at 15:46):
Also note: http://hl7.org/fhir/ValueSet/measure-type should be http://terminology.hl7.org/CodeSystem/measure-type in your examples.
Gino Canessa (Apr 06 2020 at 15:47):
No worries. I was just talking with @Josh Mandel , who chimed in directly :-)
Josh Mandel (Apr 06 2020 at 15:47):
Based on the QM guidance you can't have Numerator and Denominator with Type = Cohort.
I think you mean Measure.scoring = cohort? My take: scoring is an optional field, and if we're having trouble conveying counts + ratios because of the scoring value, then we could just omit the scoring element @Eric Haas . Does this make sense @Bryn Rhodes or is there something else you'd propose?
Josh Mandel (Apr 06 2020 at 15:48):
The goal here would be: convey the numerator (e.g., count of tests performed), denominator (e.g., count of positive tests), and ratio (e.g., percent positive) in the same report.
Keith Boone (Apr 06 2020 at 15:48):
I wansn't using cohort for exactly that reason, some of these are clearly ratio measures when you group them right.
Keith Boone (Apr 06 2020 at 15:49):
Can't type fast enough to keep up with @Josh Mandel ;-)
Bryn Rhodes (Apr 06 2020 at 16:01):
Measure.scoring is optional in the base resource, but it's required for communicating the computational semantics in the various profiles. Cohort is the simplest because there isn't really a score, we're just counting things. I agree there are proportion measures in there, but I was operating under the assumption that we wanted to communicate the count requirements as simply as possible. The last two FEMA reports are clearly proportion measures though, and I agree with Keith there, we should communicate those semantics.
Bryn Rhodes (Apr 06 2020 at 16:02):
If we want to also communicate the proportion semantics of the others, it's possible to do, but it means we will need to specify at least initial/numerator/denominator for each group (and that will mean that for some of the simpler ones, there will be redundancy in that statement).
Bryn Rhodes (Apr 06 2020 at 16:03):
We also need to be careful with the Patient one there, there's actually an unstated population that would need to be the initial population for some of the measures (covid patients in inpatient beds or other locations combined).
Bryn Rhodes (Apr 06 2020 at 16:04):
And I still don't think the basis is quite right for some of these, but I haven't ever gotten a response to the question of whether I can add a basis row to the spreadsheet. Without that we can't really express what we're saying here.
Josh Mandel (Apr 06 2020 at 16:18):
I couldn't follow this, but hopefully we can talk through examples here.
Josh Mandel (Apr 06 2020 at 16:18):
(Or on call.)
Josh Mandel (Apr 06 2020 at 16:26):
BTW submitted FHIR#26695 to address the type-vs-scoring confusion
Gino Canessa (Apr 06 2020 at 17:08):
Ok, posted a pass for review/discussion based on the call earlier. I've changed the scoring to proportion and added a denominator of 1 where appropriate.
Eric Haas (Apr 06 2020 at 22:44):
yes I meant scoring and this is my version of the measures and measurereports based off of Gino's work and looking at the source material:
- there are 3 measures and resulting measurereports and bundles.
| Measure | MeasureReport | Bundle |
|---|---|---|
| cdc | cdc-mr | cdc-bundle |
| fema-cohort | fema-cohort-mr | fema-cohort-bundle |
| fema-proportion | fema-proportion-mr | fema-proportion-bundle |
Josh Mandel (Apr 06 2020 at 22:48):
From today's call, didn't we decide to collapse all FEMA measures into a single MeasureReport? (Or... I got this wrong in my notes.)
Eric Haas (Apr 06 2020 at 23:06):
I admit just looked at the FEMA and CDC templates. My interpretation of measures is you can't mix and match since max scoring is one and max type is one and the FEMA template is a jumble. Anyhow, I need to be honest this was a slog getting to this point and in the end you have to write code to create the MR by basically by reading the measure. which I did here:
:notebook: https://mybinder.org/v2/gh/Healthedata1/MyBinder/master
Not trying to be heretical but what is the benefit of this over a plain old csv.
Josh Mandel (Apr 06 2020 at 23:09):
Not trying to be heretical by what is the benefit of this over a plain old csv.
I assume you mean "but what is the benefit"?
Josh Mandel (Apr 06 2020 at 23:11):
This is a question we should have a really crisp answer to, @Eric Haas. (I keep asking the same question myself.) I don't think we have a crisp answer yet. But here's my attempt...
I think the benefits only accrue when dealing with change, aggregation, and automation. Change, as individual organizations update the definitions/requirements for what's being collected -- because management based on CSV column names gets brittle. Aggregation implies a consistent way to bring together measures across different reports. I think these are both modest benefits, and probably don't outweigh the complexity costs.
For me the primary benefit is Automation: CSV is not an API; FHIR provides an end-to-end solution for modeling, submission, and subsequent query.
Eric Haas (Apr 06 2020 at 23:26):
thanks for the rationalization. I think the measure authoring is really the sticky part, even after reading the QM IG I'm afraid we've arrrived at totally different places.
Keith Boone (Apr 07 2020 at 01:18):
My crisp answers. 1. If you can automate MeasureReport from Measure (especially if we provide tools to enable that), getting to CSV or other formats from MeasureReport is easy. Even if you just develop MeasureReport, getting from/to other formats is pretty straightforward.
As a way to define the content, this is the kind of detail that we don't get out of CSV except perhaps from NHSN who's been doing this for so long they provide all that detail in text. 2. FHIR enables us to define push and pull APIs to access the material, and ways to convert to other formats. 3. The way we've started to write Measure enables automation.
There's going to be at least three more reports someone is going to want (if they haven't already) by June... PPE, Staffing, Immunity Status of Workforce ... and it goes on from there ... eventually, there will be immunizations, who gets it, who has it, who needs it? How/where should it flow? What data is needed to answer that question.
If these are to be deployed quickly, they need to be developed in such a way as to support automation. And they need to be easy to adapt to existing information flows.
Keith Boone (Apr 07 2020 at 01:19):
That's a lot of words, and I should really spend the time to edit them to make them all more clear. But I think you get the general idea. As I said on a call earlier ... this is going to be with us for 18-24 months, even if not at this level of crisis.
Robert McClure (Apr 07 2020 at 15:38):
Just an observer since I am not attempting measure creation here but what really worries me is the same thing @Eric Haas noted - it was not clear how to define the measures based on a spread sheet that I think is clear, at least clear enough to expect every hospital to report independently. Maybe that was because we had a round hole for a previously unrecognized square peg use case, or maybe we've not documented things clear enough. Any idea which?
Josh Mandel (Apr 07 2020 at 15:58):
I think it's more like we have a drill press and then various bits and surfaces that we can use to create holes of practically any shape using the FHIR specification -- which is where we're getting stuck. And what we want out the other end is only modestly while defined.
Eric Haas (Apr 07 2020 at 16:33):
This was a great experience in applying the Measure and MR to a real world use case and it exposed some serious shortcomings. I think the easy stuff should be clear and easy to define a Measure for and its not. For example The CDC csv is a half a dozen counts and is IMO well defined (while the FEMA template is a not). It was not immediately apparent to me to use cohort and list them as groups. Then there is the table in the QM guide that I followed and defines the pop type codes to use. Then @Josh Mandel suggests to ignore scoring and type and glom everything into one measure. I am not saying one approach is better than the other but by allowing these differing approaches then we are fooling ourselves into believing "you can automate MeasureReport from Measure (especially if we provide tools to enable that),". Instead you have to read each measure and code to it. IMO, That is where we are today.
Hans Buitendijk (Apr 08 2020 at 02:25):
@Keith Boone , @Gino Canessa , @Michael Donnelly , @Bryn Rhodes : I updated the google spreadsheets to separate the common data at the Measure level from the data at the Measure.group level (https://docs.google.com/spreadsheets/d/1TS_r8gfjZPXREOvEJaAzK-rweh8whrSH9OM6ZTA4yss/edit#gid=0) for CDC. Will do FEMA next, but in the mean time would like feedback as this is not completely in sync with the example .json yet where I would suggest adjustments to the example .json and where we have some choices. Would like to highlight those tomorrow to then ponder.
Josh Mandel (Apr 08 2020 at 04:02):
Then Josh Mandel suggests to ignore scoring and type and glom everything into one measure.
Hey, don't saddle me with that decision :-) I was just documenting in this channel the discussion we had on the phone.
Eric Haas (Apr 09 2020 at 15:47):
I know you are the messenger. It just carried more weight when I attached it to you. :-)
Keith Boone (Apr 10 2020 at 05:37):
@Eric Haas Back in the days of QueryHealth and HQMF Release 2, I developed three different proof-of-concept implementations for automation of measures from the HQMF 2 measure definitions as part of that QueryHealth project. I can claim to have some experience in this space. We demonstrated one of them using data from I2B2. The biggest challenge in automation is mapping from institution local codes to the code sets used to define the expressions that the measure evaluates. The other challenge is ensuring that the institution's data model matches the model expected by the measure. Yes, there's always going to be adaptation, but CQL actual makes a lot of that simpler in that you can isolate the adaptation pieces in the measure logic. Location identification is clearly going to be a huge challenge, because right now, the "type" of a location is a lot more dynamic than it used to be. Even so, that challenge can be factored into the measure in such a way that it can be addressed in a number of different ways. A number of organizations are already computing measures directly from CMS measure definitions from collections of CCDA data. There's certainly customization, but it's not "coding to each measure". It's more like configuring each measure for the institution, and that's a different skill set than "Code to each measure".
sean zitello (Apr 13 2020 at 18:12):
I am a bit confused by MeasureReports which contain 1 group for each population measure and the Measure-SANERCDCReporting which has 3 groups defined comprised of selected population measures. Are the user cases different for Measure-sanerCDC. Is one intended for creating the MeasureReport and the other for reading and create aggregated reports?
Josh Mandel (Apr 13 2020 at 20:10):
Can you point with deep links to some of the examples you're referring to? e.g. on github you can point to a specific group like this. Pointing to docs or examples will help folks understand and respond.
sean zitello (Apr 13 2020 at 21:22):
Good to know: https://github.com/AudaciousInquiry/fhir-saner/blob/master/docs/Measure-CDCReporting.json#L112-L468
Has Group with 3 elements, each with different number of pop measures.
Keith Boone (Apr 13 2020 at 22:19):
Some of that was left over from a prior local build, so I removed it. Need to get that automated or removed all together.
Keith Boone (Apr 13 2020 at 22:20):
We still have too many "measure definitions" for CDC and FEMA, but I hadn't gotten a chance to compare Gino's work with what I had been doing over the weekend. I will do that comparison after I get my example generator applied again.
sean zitello (Apr 22 2020 at 17:13):
Measure and references to the source of the numerator and denominator are lacking(?)
There does not seem to be a direct link inside the Measure between a (population) measure and the numerator. How do I know how to calculate the score if I don't know the source item. It can be inferred by a human, but does not appear to be deterministic by machine.
Bryn Rhodes (Apr 22 2020 at 18:15):
The Measure groups are intended to be self-contained descriptions of the calculation, with semantics described by the scoring, so each population group specifies the initial-population, numerator, and denominator (for a proportion score, for example).
sean zitello (Apr 22 2020 at 18:19):
"code": {
"coding": [
{
"system": "http://terminology.hl7.org/CodeSystem/measure-population",
"code": "denominator",
"display": "Denominator"
}
]
},
"criteria": {
"description": "FEMA defined field: cumulativePercentPositiveAmongResultedTests",
"language": "text/plain",
"expression": "# of total test results released to date."
}
Here for instance: expression value does get me to a specific item in this Measure.
I would expect to find some reference to this item for instance:
{
"code": {
"coding": [
{
"system": "http://build.fhir.org/ig/AudaciousInquiry/saner-ig",
"code": "cumulativeTestsPerformed"
}
],
"text": "All tests with results released to date."
},
Bryn Rhodes (Apr 22 2020 at 18:34):
My understanding is that the focus currently is on optimizing consumption of the MeasureReport, not necessarily the computable representation of the Measure. I'm keen to help figure out how to support both, but I haven't been able to join any of the calls so far (they've all been at times I haven't been able to join).
sean zitello (Apr 22 2020 at 19:59):
Ahh. OK. my goal is to build Measure to MeasureReport engine so to speak. What I have done is just change the 'expression' to refer directly to the code. I have seen some examples using fhirpath, i should examine that a bit, but if all the factors are either static or contained with this Measure def, then its pretty robust. All I really want is the code name and if it more complicated than numerator/denominator, I'll learn and adjust.
Eric Haas (Apr 22 2020 at 19:59):
It can be inferred by a human, but does not appear to be deterministic by machine.
You are correct .... all measures are text description based on the FEMA CDC descriptions. The MeasureReports are created by somebody reading the measure (definition) and coding to create the report. In more advanced measures the criteria can be defined for a machine to read them and spit out code for a MeasureReport.
sean zitello (Apr 22 2020 at 20:06):
ok thanks. just wanted to understand what the standard expects and what I should expect. In truth I am not going to go get the Measure off the server at the time the MeasureReport gets generated. I'll cache and add those little edits to it and the machine will be happy along with me.
Keith Boone (Apr 23 2020 at 12:03):
Interestingly, I create the MeasureReport examples directly from the Measure itself, and while right now, am reading the measureScore from the input data (e.g., a CSV file), do know how to compute it from the parts of the measure. You want to look into the Quality Reporting topic: https://www.hl7.org/fhir/clinicalreasoning-quality-reporting.html which is a link inside Measure and MeasureReport. Sadly it doesn't get any love on the FHIR front (index) page (it should probably be listed in Clinical Reasoning section, or perhaps even deserve a section of its own).
Grahame Grieve (Apr 23 2020 at 12:34):
you're welcome to create a task about that
Keith Boone (Apr 23 2020 at 13:11):
Added: https://jira.hl7.org/browse/FHIR-26931
Abbie Watson (May 07 2020 at 14:28):
So, I'm happy to report that we have the example Measures and MeasureReports loaded up with user interface, and are on track to send/receive at the connectathon next week. The MeasureReports are working great. The Measure definitions themselves, though... eh? It's somewhat obvious that they're being used in a different context than the quality measures we received from, for example, the College of Cardiology for their hospital accreditation process. I can adjust the UI easily enough, but am debating the best way to go about that.
Gino Canessa (May 07 2020 at 15:40):
That's the kind of feedback we want (generally and specifically from the connectathon). There are many ways of structuring the data, so if things don't make sense, or even would just work better a given way, we want to know. With feedback from multiple implementers we can (hopefully) land on what is the best model across the known use cases.
Abbie Watson (May 07 2020 at 16:36):
I'm aware. Been using HL7 interfaces for 17 years, and this is my 8th or 9th FHIR connectathon. I'm trying to get the feedback in early, because I'll also be participating in Gravity and BlueButton/DaVinci tracks also. Thanks.
Gino Canessa (May 07 2020 at 17:10):
Sorry if that came across wrong - I've been worried that there are only a couple of us giving model feedback (so far) and was hoping for specifics around what isn't "right" in the current models for discussion. Just didn't want it to get lost if you adapt on the other end.
sean zitello (May 08 2020 at 21:05):
The new Measure CDCPatientImpactAndHospitalCapacity is structured differently than it was for sanerCDC. For sanerCDC each Measure was basically its own group, each item was an initial-population, etc. Newer version has 3 Groups with the first group item initial-population none of the others and without going too far down the rabbit hole, the Question is whether than was the intent. It explains all the empty data because we need to process the group element in a different way. I don't want to go to too much effort changing that if thats not the intent of the changes.
sanerCDC: image.png
CDCPatientImpactAndHospitalCapacity: image.png
The second is more like what we had seen early on and I think choose to simplify with sanerCDC. I may have missed some decisions along the way, so perhaps agreed to revert to this, but I'd rather be sure before I start rework.
Keith Boone (May 09 2020 at 13:44):
Yes … you did miss some decisions along the way. Measures are grouped by "What they count".
John Moehrke (May 12 2020 at 00:41):
I am noticing that there is no defined Measure.topic or Measure.type defined... I would expect SANER to have picked one of these to indicate for this use-case. Otherwise I am not sure how to query for CDC Bed topic. I can't query by group statifier
Abbie Watson (May 12 2020 at 05:00):
Here's a revised version of the CDC Measure that displays nicely in our Cardiac Accreditation utility (which also happens to use MeasureReports). I added type, author, editor, reviewer, scoring, group descriptions, critiera names, and critiera references.
https://github.com/symptomatic/covid19-saner/blob/master/data/measures/Measures-CDC-Revised.json
Abbie Watson (May 12 2020 at 05:19):
And a revised FEMA Measure:
https://github.com/symptomatic/covid19-saner/blob/master/data/measures/Measures-FEMA-Revised.json
Josh Mandel (May 12 2020 at 13:51):
These measures look pretty different from the ones in the implementation guide, or at least at https://github.com/microsoft-healthcare-madison/learning-spike-erp/blob/master/generated/t0/Measures-CDCPatientImpactAndHospitalCapacity.json ; is this expected/intentional? For example are you proposing changes or moving in a different direction? Or is it just a question of having time to sync everything up?
John Moehrke (May 12 2020 at 13:54):
so it seems .useContext is being used rather than .type or .topic. I think I understand the .useContext and there is a query parameter context that can be used to find all the Measures for covid-19.. but can't be used to differentiate the CDC measures from the FEMA measures. Still seems we need to have a differentiation between the two.
Josh Mandel (May 12 2020 at 13:58):
That's true -- and I'm primarily focused on the structure of the groups and whether information is being conveyed in population versus measure scores.
John Moehrke (May 12 2020 at 14:02):
teamwork
John Moehrke (May 12 2020 at 17:26):
okay, so I have been schooled by Keith. I really was not understanding that "Measure" resource and our use of it is as a definitional resource (should be called MeasureDefinition). Now that I know that, I have learned that I should be looking at MeasureReport, and that the answer to my differentiation is to be found in MeasureReport.url as either the canonical URI to the CDC measure definition, or the canonical URI to the FEMA measure definition.
Jose Costa Teixeira (May 12 2020 at 17:27):
John Moehrke said:
should be called MeasureDefinition
+1
John Moehrke (May 12 2020 at 17:49):
Abbie Watson (May 12 2020 at 17:51):
@Josh Mandel - I was using the measures from an earlier version of the implementation guide. What Gino had published a week or two ago. And simply running them through our software. I think my main points of contribution is that group.description and criteria.name should be filled out, as well as the author/editor/reviewer workflow.
Keith Boone (May 13 2020 at 04:43):
Yeah, the existing example definitions for "topic" are hard to apply. They are focused on patient associated processes, treatment, education, assessment. It's hardly clear what the right set of classifiers are yet for measures. When you have 100 measures, it's pretty easy to figure out how you want to cluster them. When you have four or five, four of which are just mirrors of the same measure with different "treatment services" (e.g., Acute, Long-Term Care, Rehab, Psychiatric care), but others are very differently focused.
As for Measure type, well, in fact, we are working with MeasureReport more as a "report card" on an institution, which might include several groups and populations, with each group potentially having a different "type" of measure. Consider an academic report card, in which a student gets a letter grade for certain subjects, a pass/fail for others, and feedback on number of days present or absent. That's ONE report card, four different groups of data, and yet still "One report", not a "bundle of separate reports" just because the types of each report are different.
So, we created an extension on group to capture "measure type" at the group level. And, yes, that will be feedback on R5. Because doing it that way won't critically impact existing quality measures in any way that we can determine. They'll still be able to be adequately defined with the same concepts, but we have a little bit more versatility in MeasureReport which allows it to be used and communicated more efficiently.
Last updated: Apr 12 2022 at 19:14 UTC
