FHIR Chat · value for an unsupported population · Covid-19 Response

Stream: Covid-19 Response

Topic: value for an unsupported population

Michael Donnelly (May 13 2020 at 05:32):

If you get a MeasureReport that includes a count for a population you don't support (in a group you do), what should you do?

My thought is that having some data isn't as good as all but is better than none, so you should file it. Anyone have other perspectives?

Keith Boone (May 13 2020 at 10:14):

File with a data-absent-reason. That's what our csv-to-measureReport coping mechanism does.

Michael Donnelly (May 13 2020 at 13:45):

In my case, the server doesn't have anywhere to file it. When we produce a MeasureReport, we have a DAR for elements that are absent (unsupported for populations we don't support at all, unknown for supported populations for which we don't have data).

Keith Boone (May 13 2020 at 18:06):

Yeah, I mean, when you produce it, produce with DAR...
someone else will file it.

Michael Donnelly (May 13 2020 at 21:29):

So when I receive a MeasureReport and the report has populations that I don't support, my options are

1. Error out, reject the document.
2. Store the populations I can, ignore what I can't.

How would you folks handle this?

Gino Canessa (May 13 2020 at 21:35):

Does this MeasureReport point at the canonical Measure?

Gino Canessa (May 13 2020 at 22:17):

Capturing discussion on Zoom:
The scenario is that one of the fields is NOT able to be stored in the server's data model. Because of this, any MeasureReports received by the server will have that value stripped or be rejected.

So perhaps this is a conformance question? If you cannot store data for a specific field, how do you indicate that to clients and what is the behavior if they try it anyway?

@Michael Donnelly Does that summary sound right?

Gino Canessa (May 13 2020 at 22:27):

Discussion led to:
Intermediaries have an obligation to ensure that no data is lost. For anyone else, you get what you get.
Statement in the Grouping Behavior section?

Abbie Watson (May 13 2020 at 22:31):

We have incoming validation turned off, which is something that a security minded person might complain about, but which is pretty typical in the networking world. Just doing a proxy/relay. I'm more concerned/interested in aggregation (agenda item to discuss?).

Abbie Watson (May 13 2020 at 22:33):

I've been thinking of it similar to voting systems, where the tallies come in from the different hospitals and cities and precincts and counties, and sort of aggregate next to each other. Whether or not to assign slots... that's what I'm curious about. Do we do a Health Service Area approach, or a free-form approach (or both).

Michael Donnelly (May 13 2020 at 22:34):

Unless I misunderstand, I think what you're talking about is slightly orthogonal to my concern, @Abigail Watson .

Michael Donnelly (May 13 2020 at 22:36):

What @Gino Canessa, @Reece Adamson, @John Moehrke and I figured out on the teleconference is that (we think) there are two cases:

Michael Donnelly (May 13 2020 at 22:39):

1. A Measure Intermediary intends from the beginning to collect, aggregate, and retransmit reports. It would be failing in its duty if it doesn't pass on information it receives (except in cases of deduplication or other good reasons).

Michael Donnelly (May 13 2020 at 22:40):

2. A Measure Consumer receives a MeasureReport, stores (what it can) in its (non-FHIR modeled) database, and uses those data. At some later point, the same system acts as a Measure Source, and some or all of the data it shares happened to have made it into the system via the previous MeasureReport. The fact that it doesn't include data it happened not to have a place to store is fine.

Abbie Watson (May 13 2020 at 23:14):

I grok the functional description, but am compelled to ask 'why'? Why are we consuming the measures? For what purpose? Is it so we can allocate resources? Or is it for state bragging rights? Are we collecting measures so we can open economies sooner? Or because we need to know how long to continue self isolation? Arguably, it's all of these things. But if we understand the purpose, I think we have a better understanding of what we should do with counts for populations the system wasn't anticipating.

Like... is this a zero-sum game? A prisoner's dilemma? Or a socially cooperative game? The answer to those questions will suggest what to do with those unaccounted for populations. (imho)

Michael Donnelly (May 13 2020 at 23:15):

Good point.

Michael Donnelly (May 13 2020 at 23:15):

And thinking about it reveals that I haven't asked myself "why?" enough times.

Michael Donnelly (May 13 2020 at 23:17):

Mmmm. Yeah, okay, good enough question that I know I won't stand a chance of coming up with a good answer if I don't eat dinner first.

Abbie Watson (May 14 2020 at 00:50):

So, a bit more from the geomapping side of things.... we take the premise that pandemics are fundamentally about population density and geospatial in nature, and therefore beg to be visualized with maps.

Okay, but what kind of maps? I bring up the questions about zero sum games vs socially cooperative games, because I think a) we have the current mandates because we (society, congress, etc) want self-correcting feedback loops in place, b) maps provide actionable artifacts that people can make decisions on, so c) we're heading towards some sort of gamification or forecasting usages of this data. Whether that gamification is government reports, meteorologists updating their maps, or some sort of dystopian Hunger Games, I don't know.

But even within that premise, there's a bunch of different ways to go about it. I bring up zero-sum games vs socially cooperative games, because the maps we create can be zero-sum bounded or open-ended. That's the difference between a choropleth and heatmap. Choropleth maps account for every square inch of space in the US. Every square foot of land gets assigned to a county, state, health service area, or some other region. (The IPPS data file that Josh Mandel found was particularly useful in that regard, because not only does it have the CMS hospital numbers, it also provided the cross-walk to the Health Service Area (HSA) shapefile regions.) Regardless, that boundedness makes it zero sum, in so far as real estate is fixed and the counts have to be applied to a region in order to be visualized. If you want to cross reference census data with it, you can zero-sum it further with a cap of ~330M citizens, and calculate a percentage for each region in the choropleth.

Alternatively, an heatmap is open-ended. Have illegal immigrants or undocumented populations? They just increase intensity on the map. But you can't really do a county-by-county quarantine. But you could calculate risk based on proximity to other hotzones.

We can do either approach. They both have pros/cons, and offer something a bit different. But for a choropleth map with bounded inputs and a zero-sum approach, the unknown populations necessarily need to get grouped in with existing regions (or somehow eliminated/removed - a distressingly unpleasant and very problematic notion in the context of healthcare and pandemic). For a heatmap, they simply need to be harmonized and avoid double counting.

Also, leaderboards. Same problem, different dimensions. When this is all in place, there will naturally be a leaderboard. Is it the hospitals? The counties? The states? Is anybody allowed to participate? What about Puerto Rico? Or the international space station? Are we only allowing hospitals with CMS numbers on the leaderboard (zero sum)? Or do we allow any field hospital to register itself? What about the Navajo Nation? My contacts in the National Parks Service is saying that things are real bad on the reservations, many of which don't have running water so soap doesn't work even if they have it on hand.

$0.02

Michael Donnelly (May 14 2020 at 01:28):

I'm way too ignorant about geomapping to contribute anything useful in that space. (Like, I just had to look up "choropleth.")

What I *can *do is make sure I don't do anything to damage the quality of the data that pass through my system's hands, so I don't make it harder for people who need to make use of it later.

John Moehrke (May 14 2020 at 12:54):

What we came up with is a clear distinction between a system that is intentionally an intermediary, and when an intermediary they must not discard any data that is conformant to SANER, and they must represent that data with appropriate quality and algorithm on the Measure Source actor (trying to get words that allow an intermediary to do math on multiple inputs, remove duplicates, etc.. as an intermediary adds value)... thus if you want that kind a system, you have the intermediary actor to call for.
Where as if a system just happens to have implemented both a Measure Consumer and a Measure Source... but not intermediary... then there is no clear requirement or expectation that it will channel ANY of the data from input to output. It likely will channel some, but one needs to look at the app functionality to determine what it might do to the data.
What could this system possibly be doing? First, the inputs might be totally unrelated to the outputs. The functionality is simply not there. Second the system may only be handling a fraction of the SANER functionality as an intermediary (it is a value-add intermediary for bed count ONLY).

John Moehrke (May 14 2020 at 12:56):

Are we requiring that a Measure Consumer must process all the measures? What if that system implementing a Measure Consumer is a dedicated visualization of bed availability ONLY. It has no use for the other values. Are we requiring that Measure Consumer to visualize every measure? I don't think so.. so this is simply what the understanding is based upon.

John Moehrke (May 14 2020 at 13:04):

having also needed to lookup choropleth (even chrome doesn't like the spelling, love its suggestion of plethora)... @Abigail Watson I think you are way outside my subject-matter-expertise. I am trying to get a feel for the impact on the scope of SANER. The Location regions seem critical to what you are describing. However SANER doesn't define those Locations, we enable them to be defined. So, to define the shapes as overlapping, non-overlapping, consistent with other regions, etc.. is a policy decision outside the scope of SANER.

Andrea Pitkus, PhD, MLS(ASCP)CM, CSM (May 14 2020 at 13:57):

@Abigail Watson Great questions. Some of my federal contacts have indicated FEMA reporting (lab testing, vents) is sent to the WH/FEMA Taskforce, then disseminated to other federal (FDA, ONC, CDC, FEMA, to name a few) agencies as they are all working on different aspects of the COVID-19 response.

Josh Mandel (May 14 2020 at 14:13):

Looking through the discussion above @Abigail Watson, your assessment implies we are mapping vsomething like infection rates normalized by population or density... But so far all the measures we are looking at deal in "simple" counts or percentages of patients in a hospital where the denominators are known. (Possibly I am just missing something!)

Abbie Watson (May 14 2020 at 15:15):

I bring the geomapping stuff up, not because I expect everybody else to be mapping experts or nitpick the details of map implementation, but to share downstream usecases of Measure Consumers and to provide concrete tangible algorithms to refer to. So, for the report out, please feel free to reference the following:

** 

Geomapping Use Case: Choropleth

** Geomapping Use Case Heatmap

And, in particular, I think we need to pay close attention to the rules around MeasureReport.type and MeasureReport.group.population.code values. 

So, for instance, this is what’s happening on the Intermediary and Consumer.

MeasureReportConsumer(MeasureReport.type=‘subject-list’) => Heatmap
MeasureReportConsumer(MeasureReport.type=‘summary’) => Choropleth
MeasureReportConsumer(MeasureReport.type=‘individual’) => Simplemap

And to Josh’s comment, I would posit that all geomapping algorithms necessarily involve mashing up external data. At the very least, there’s some trigonometry involved with the spatial coordinates. And it quickly grows as the cartographer adds geography, census data, economics, meteorology, etc. That’s the art of cartography! SANER is simply the way that the variables get fed into the mapping algorithms (from the Consumer’s point of view). 



As far as SANER scope goes, I’d like to see some language like the following…



A MeasureProducer that generates a MeasureReport of type summary SHOULD set the MeasureReport.group.population.code as initial-population.

An Intermediary that combines multiple MeasureReports of type individual SHALL update the type to summary.

An Intermediary that combines multiple MeasureReports SHALL preserve all data elements. (wording?)

An Intermediary that combines multiple MeasureReports of type ‘individual’ SHOULD set the MeasureReport.group.population.code to a value of measure-population.

A MeasureConsumer that consumes MeasureReports of type summary SHOULD use a choropleth (MapType ValueSet TBD)

Keep in mind, that even adding this stuff in, there will remain plenty of policy and political factors outside the scope of SANER. Even with these rules, choropleths can be produced to advocate for both continued lockdown or reopening.

Terrie Reed (May 14 2020 at 15:20):

I have a question about what systems (or manual processes) are the assumed sources of the data that will result in number of beds and ventilators/hospital. Depending on hospital, the ventilator inventories could be managed in Respiratory Therapy and Biomedical Engineering and hospital item master or one or none of the above. I'm curious if there are documented assumptions being made about these data sources?

Abbie Watson (May 14 2020 at 15:21):

And to get back to @Michael Donnelly's original question, anything involving unaccounted for groups and populations, is (in my mind) a qualifying event to updateMeasureReport.type and MeasureReport.group.population.code.

Chris Turczynskyj (May 14 2020 at 15:27):

(deleted)

Last updated: Apr 12 2022 at 19:14 UTC