Stream: bulk data
Topic: Exporting a group subset
Nick Robison (Jun 03 2019 at 20:16):
Hi folks!
Over at CMS, we've been working on a pilot project exploring some ways of using the Bulk spec to more closely integrate with EHR systems. As part of the pilot we've run into a use case where we'd like to support exporting a subset of an existing group. For example, exporting all the patients for a given organization that have an appointment in the coming week.
Internally we've been discussing how best to implement this.
One thought was to add a POST method to the $export operation (e.g. POST Group/1/$export) with the user submitting a Group resource which would contain a subset of patient IDs.
We've also talked about using a query parameter to the defined GET endpoint, as well as forgoing supporting this feature at all, in favor of the traditional FHIR endpoints.
Any thoughts on a preferred or more FHIR-like approach?
@Rick Hawes can also chime in with some of his thoughts, he's been working on our implementation as well.
For those who are interested, the code is publicly available here: https://github.com/CMSGov/dpc-app
Josh Mandel (Jun 03 2019 at 20:28):
POST method to the $export operation (e.g. POST Group/1/$export) with the user submitting a Group resource which would contain a subset of patient IDs.
One possibility is to allow the client to define ad-hoc groups (by POST /Group, or if you want to enable some extra requirements like strict subsetting, or client-supplied demographics, maybe a custom operation like POST /Group/:supsesetId/$subset-create, which takes a Bundle of Patients to include in the subset).
Then once the subset group is defined, treat $export on it like normal.
Paul Church (Jun 03 2019 at 20:33):
In the ideal world, does the client want to supply the subset by knowing their resource IDs up front or does it want to identify them through a search?
Nick Robison (Jun 03 2019 at 20:47):
I think an ideal world would support both use cases, for our system, we don’t necessarily have the information required in order to create some groupings.
Paul Church (Jun 03 2019 at 21:32):
POST /Group/supersetId/$subset-create?search-query sounds like a reasonable custom operation.
Is _typeFilter flexible enough to do a GET /Group/supersetId/$export?_typeFilter=Patient%3Fsearch-filter-for-subset? I guess that doesn't filter the non-patient resources in the export...
Nick Robison (Jun 04 2019 at 12:13):
I like the idea of the POST operation on the /Group endpoint. So the endpoint would be POST Group/$export, would that be an acceptable solution? If possible, I'd rather not require a two-step process to generate a transitory group (which may be discarded immediately after the operation completes) and then export the data. It would be nice to be able to generate an ad-hoc group within a single operation and not require that temporary data to be persisted.
My concern with using _typeFilter is two-fold. One, we're not planning on adding that feature until a later release as we'd like to roll out support for complex queries in a safe and performant way. Two, in our use-case we don't necessarily have the information required to return the correct subset of patients. In our example, we don't know which patients are scheduled for an appointment, which means the clients would have to construct a query with a large number of patient ids, which feels a bit odd.
Josh Mandel (Jun 04 2019 at 12:39):
If you have a one shot operation two export data on a temporary group, where do you manage information about being able to say which patients in the opposed temporary group matched or did not match, or whether authorization errors were generated?
Josh Mandel (Jun 04 2019 at 12:40):
Also, would clients want to continue getting updates about this same group overtime? If the use case include this kind of functionality, then persisting it would seem to make sense.
Nick Robison (Jun 04 2019 at 12:50):
For the patient matching, our draft implementation returns an OperationOutcome which lists the patients that either did not match or that the client is not authorized to retrieve data for. I'm sure there are some instances where the client would like to retrieve updates to a patient group over time, but I'm not entirely sure that functionality should be the domain of the server, rather than the client. Wouldn't it make more sense for the client to maintain a list of patients groupings that it's interested in and simply export on the ones it needs data for?
Josh Mandel (Jun 04 2019 at 12:57):
I guess it depends how long the list is and how often the client wants updates. Mostly I'm trying to think about building blocks here -- components that would be useful/reusable in a number of limitations.
Josh Mandel (Jun 04 2019 at 12:58):
Currently when you return an operation outcome, you do that synchronously? And also kick off an export job for the patients that were matched and allowed?
Nick Robison (Jun 04 2019 at 13:17):
We're currently returning the OperationOutcome synchronously, but we're not submitting the job unless all the matching/authorization succeeds. This is part of our 'fail fast' mindset, if we know we can't complete the job as specified, we fail it and let the client handle the errors. This is something we're hoping to get feedback on from early users.
Josh Mandel (Jun 04 2019 at 13:20):
Gotcha -- so you're only evaluating matches that you know you can do reasonably quickly for a synchronous response.
Josh Mandel (Jun 04 2019 at 13:21):
But even in a case of a strict subset, you are not expecting a client to know the FHIR IDs of the patients in the subset? In other words the client is not just supplying IDs but a complete patient resource as the input to the subset/match?
Grahame Grieve (Jun 04 2019 at 13:28):
I think it's important to think of this as a subset, so that servers can manage expectations
Josh Mandel (Jun 04 2019 at 13:32):
Agreed Grahame -- does that take you in one direction or another with the design?
Grahame Grieve (Jun 04 2019 at 13:34):
no. I was catching up on something I meant to say earlier
Nick Robison (Jun 04 2019 at 13:46):
In our case, we do have a stable identifier that we can use (medicare number), so we're requiring them to submit a group with the given identifier. However, I realize not all systems have that benefit. We're still considering ways to support searching for patients given a set of demographic information, but that's not on our immediate roadmap.
It seems to be me have two use-cases.
1. The ability to execute ad-hoc export operations with a temporary grouping of resources that the client has knowledge about.
2. The ability for the client to define custom groupings of resources that can be continually queried over time.
I believe the 2nd use case is already handled with existing endpoints, in that we know how to create and persist fairly arbitrary groupings of patient through the existing GROUP endpoint. But it's the second use case that doesn't seem to be supported.
Josh Mandel (Jun 04 2019 at 13:46):
But number one also includes the fact that it is a subset of an existing group?
Josh Mandel (Jun 04 2019 at 13:48):
I think you're referring to using a specific system and requiring Patient.identifier - rather than Patient.id here. I bring this up just because typically when we provide a list of references, the way to do this in FHIR is via id.
Josh Mandel (Jun 04 2019 at 13:49):
In your existing implementation do you use a mixture of query perimeters and body to send along the details for the ad hoc export request?
Nick Robison (Jun 04 2019 at 14:02):
You're correct, I'm conflating Patient.id with Patient.identifier, apologies. It seems the patient.id would be an implementation detail of the system, are we required to leak that to the clients? I must be misunderstanding the reference resource, are you saying it can only reference the internal ID of the resource, not a common Identifier such as NPI?
Josh Mandel (Jun 04 2019 at 16:27):
You can create a reference by id (the most common case, populating reference.value) or by identifier (populating reference.identifier). Both of these are different from directly including a patient resource in a bundle
Nick Robison (Jun 05 2019 at 12:55):
Hi folks, I think I did a poor job explaining our problem and solution yesterday, my apologies.
I've updated our feature PR on Github to give a bit more background and example data. You can view it here: https://github.com/CMSgov/dpc-app/pull/58
Let me know if anything is unclear or seems off. I've tried to implement some of the feedback and insights from yesterday.
Josh Mandel (Jun 05 2019 at 13:06):
Thanks Nick! The context is helpful, and I shared some initial feedback on the PR. Basically: POST /Group/:id/$export is already defined, so it is not something that we can entirely reassign the meaning of.
Josh Mandel (Jun 05 2019 at 13:07):
But there is room to play if the body is a Parameters resource.
Josh Mandel (Jun 05 2019 at 13:08):
I'd look carefully at breaking this into two steps: one define-subset step, which creates a group, and Then one optional extra parameter on $export like limit-to-subset which takes a reference to a group.
Josh Mandel (Jun 05 2019 at 13:10):
But if you want to pass in a group directly, you can do this with the existing $export, if you
1. Call via POST
2. Pass a resourceType: Parameters body, including all the existing parameters currently defined
3. Add an optional extra parameter like limit-to-subset That takes a group by value instead of by reference
Michele Mottini (Jun 05 2019 at 13:44):
Why not just create a group with the normal POST .../Group operation and use it in the standard way?
Josh Mandel (Jun 05 2019 at 13:44):
That works fine if you do not care about enforcing the specific subset semantics. Definitely where I would start.
Josh Mandel (Jun 05 2019 at 13:45):
At the authorization level, @Nick Robison you could just ensure that any organization creating a group only included patients they were authorized to include.
Michele Mottini (Jun 05 2019 at 13:46):
I'd check permissions when exporting, not when creating the group - seems safer / easier
Josh Mandel (Jun 05 2019 at 13:57):
Both are important -- creation time especially if you want to be able to fail early, and export time because authorizations of course can change.
John Moehrke (Jun 05 2019 at 14:08):
can never check authorization too many times... :-)
John Moehrke (Jun 05 2019 at 14:09):
check early, to fail early.. check late to really be sure no policy or other factors changed between.
Last updated: Apr 12 2022 at 19:14 UTC
