Stream: implementers
Topic: Async use cases: delay vs data deluge
Josh Mandel (Jun 30 2021 at 16:47):
Today, FHIR's async support is geared toward the "data export" use case, where processing is async because a server needs to sift/collate large piles of data.
Do folks have other use cases for async?
I'm especially interested in cases where you might not have huge piles of data (in or out), but where processing might be async for other reasons (architectural decisions like work queue-driven processing, out-of-band review, etc). Use cases where you might want to asynchronously handle an individual CRUD request?
John Silva (Jun 30 2021 at 17:23):
When working with "high frequency" patient monitoring data (e.g. HR, BP, temp, etc.) that come in at 30 seconds from every bedside in a critical care unit the load (Observation) could overwhelm many FHIR servers. Maybe an async pattern (or FHIR Bulk Data) would help here.
Of course the 'flip side' of Bulk Export -- Bulk Import -- would have to handle large amounts of data and maybe that would also be suited for the async pattern (assuming the immediacy for the data to be available is 'lower priority' than current FHIR server resource data access.)
Lloyd McKenzie (Jun 30 2021 at 18:41):
CDex is supporting asynch query where there's human intervention, but we want to have an ability to communicate accept/reject including reason, current 'business status' of the request, etc. For that reason, we opted to use Task rather than the FHIR Async.
Grahame Grieve (Jun 30 2021 at 19:36):
I've seen it used where processing was going to be slow for external system dependency reasons
Morten Ernebjerg (Jul 01 2021 at 06:02):
We're looking at async for validation. Specifically, we have a use-case where clients submit potentially very large Bundles of resources to our validation endpoint and the server needs to do full validations of everything. Since we do not want to impose an upper limit on the size, we cannot ensure that validation will finish before the call times out, so we want to offer async processing.
Jordi Cabré (Jul 01 2021 at 09:42):
I absolutely agree with @Josh Mandel . Currently, only bulk is async available using header Prefer: respond-async. We're very interested into set up async processing for whichever request. Do you have any schedule in order to get it available?
Josh Mandel (Jul 01 2021 at 13:22):
@Jordi Cabré can you share examples of some of your use cases / reasons you're very interested? This conversation is intended to help evaluate priorities.
Rik Smithies (Jul 01 2021 at 16:10):
EMA plan to use this for cases where reads are fast (cached) but multi-resource writes into back end systems may be slow. Not huge amounts of data.
Jordi Cabré (Jul 02 2021 at 11:44):
@Josh Mandel The most important issue we would like to "solve" is to enqueue requests in order to avoid time-outs.
Our FHIR repository will be requested eagerly, and we'd like to offer this solution... I think it would be very interesting to get it... What do you think about?
Jordi Cabré (Jul 16 2021 at 11:32):
@Josh Mandel Do you have any iadea about any planification about async implementation?
Josh Mandel (Jul 16 2021 at 12:53):
Not really, but I do think it's worth putting together a proposal to drive some discussion and experimentation.
Josh Mandel (Jul 16 2021 at 20:46):
OK, I put together a brief proposal at https://hackmd.io/@jmandel/fhir-async-cruds (thanks to @Gino Canessa and @Dan Gottlieb for discussion + initial review).
I'd love to get thoughts on this (FYI @Jordi Cabré @Morten Ernebjerg @Grahame Grieve @John Silva @Paul Church)
Paul Church (Jul 16 2021 at 21:26):
We haven't really run into use cases that need this - there is a Google-specific long-running operations framework that we use for a bunch of things (and could probably be tweaked to fit this pattern), but not for basic CRUDS. The performance impact of doing validation synchronously has been acceptable. Queuing up resources to submit asynchronously can be solved with a variety of intermediate systems.
The proposal is reasonable, I'm just not currently motivated to pursue it by any customer needs that I'm aware of.
Josh Mandel (Jul 16 2021 at 21:44):
Thanks for the quick feedback! (I'm also not sure whether there's a strong need; so far, trying to capture what we heard and think about how to fill the gap. It would be interesting to hear about specific long running operations that you do support internally, if there are details you can share.)
Paul Church (Jul 16 2021 at 21:55):
import, export, deidentify (of the whole store), configureSearch (reindexing), some of the create/delete store operations if we need more time to set up or clean up dependencies - I think that's it currently
ravi.kuchi (Jul 25 2021 at 21:09):
@Josh Mandel , we have been looking for a direction for Async PUT & POST requests (not bulk) for quite sometime now. In our setup we have legacy EHR and we have events that sync-up the FHIR server we have setup. All clients are hooked to FHIR server and they want to perform operations like creating an Order , Condition etc. Now the problem is since the EHR is source of truth we have to perform the operation asynchronously i.e., take the request and then negotiate with our EHR which is terribly slow. So yes, your proposal makes complete sense for our use case as well. As of today we are creating a Task resource to maintain/track the state of the request.
Grahame Grieve (Aug 17 2021 at 04:47):
@Josh Mandel I' m confused by this. The specification as documented says that the async pattern can be used on any request, so I'm not sure what you're pursuing here
Josh Mandel (Aug 17 2021 at 13:22):
The introduction of my doc tries to explain this; not well enough, by the sound of your question. Briefly: current "bulk async" pattern doesn't fit well for small data outputs and doesn't have a way to convey the full response semantics of the FHIR http API (status codes, etc). I'm proposing a fairly small tweak that retains the mechanism we have in r4 and expands it.
This is not super urgent for my perspective but we have heard interest and I wanted to capture this in a concrete proposal.
nicola (RIO/SS) (Aug 17 2021 at 16:06):
@Josh Mandel what if a client will lose the status URL? There should be discovery for statuses. Bundle.type = async & GET /Bundle?type=async ? From the other side, all of this is suspiciously similar to "Queues" protocols.
Josh Mandel (Aug 17 2021 at 17:00):
Re: lost URLs, agreed there could be an API for this. From what I recall, this hasn't come up for the existing bulk data API access, but it is an issue there too. I think some server developers may have explored Tasks for something similar.
Re: queue protocols, is there an existing standard that you would point us to as an alternative to this mechanism? This would be a great time to open up the aperture on solutions we are considering.
nicola (RIO/SS) (Aug 18 2021 at 06:35):
I was unable to find standard protocol - looks like every cloud or queue engine invented their own. I think we only need to specify posting tasks in a queue and introspecting results and as optional extension creation of queue. Potentially subscription may be used to wait for a response. We recently discussed queue as an alternative for CRUD for integration. There are concerns about how to link resources created through the queue - logical references by source identifiers? Another concern is how to keep business transaction borders - bundle? Samurai has implemented a queue in one big project - I will ask our engineers to share implementation details. @Josh Mandel what is your primary use case for async crud? Is it integration?
Josh Mandel (Aug 18 2021 at 13:40):
Thanks! If you scroll up in this thread, you can see some of the use cases that people shared; I tried to capture them in my document.
Josh Mandel (Aug 18 2021 at 13:40):
It would be great to document additional use cases if you've got others. For example perhaps some of the motivations that led to the queue based integration you mentioned.
Marat Surmashev (Aug 18 2021 at 15:39):
Hi
A little bit details about our implementation of Integration Queue based on FHIR
Our main task was to link tons of entities from different systems.
For example: Patients comes from regional insurance system. Medical information from local Medical Information System
Practitioners and Roles - from billing system
Organizations and Locations - from another government medicine terminology system
Usually, for any entities in this cases we have global identifier: for Patients and Practitioners- ssn and Individual insurance account number, for Organization - government identifier etc.
We are create custom resource named IntergrationQueue
Scope of this resource is primitive
{status: <"pending" "processed" ....>
clientId: <oauth client who create this message>
method: <"post" "put" ...>
payload: {ANY FHIR RESOURCE}}
Example: creating Patient
POST /IntegrationQueue
{clientId: tfoms
method: post
payload: {resourceType: "Patient"
identifier [{system "ssn" value "123"}]}}
When we create IntegrationQueue resource we are validate payload and check access rules (check does this client can do this action: e.g. can tmofs create Patient?). After that we are save this message as regular fhir resource and return to the client id of this message. Using this id client can check status of message GET /IntegrationQueue/<id>
Another message - create Document Reference from MIS
POST /IntegrationQueue
{clientId: MIS
method: post
payload: {resourceType: "DocumentReference"
subject {type Patient
identifier {system "ssn" value "123"}]}}
On another side we are have processing periodic job that take messages and process that (sometimes with tons of custom business rules)
So we got eventual consistency system based on logical references
Our Queue is just introspectable fhir resource
Queue stored in postgeresql table
Josh Mandel (Aug 18 2021 at 18:47):
Thanks -- this is super interesting.
Is the integration queue "general" with respect to the FHIR AP? In other words, can you submit (to the queue) any FHIR interaction that you could represent in a batch Bundle entry? Do you convey things like http request/response headers (say, etags) and custom operation invocations?
Marat Surmashev (Aug 18 2021 at 20:43):
We thought about it, but we prefer to keep it as simple as possible: one message one action one resource. This approach makes it very simple to develop and testing.
Transaction bundles, conditional operations, custom operations, etags - sometimes may be very useful, but it is too complex =)
Grahame Grieve (Aug 19 2021 at 07:48):
ok I think this pattern makes sense
Josh Mandel (Aug 19 2021 at 13:21):
@Grahame Grieve can you clarify whether you are referring to the async writeup I shared or to the queue-based processing model Marat described, or just the general use case of better support for asynchronous stuff
Grahame Grieve (Aug 19 2021 at 20:12):
the write up you shared
Christiaan Knaap (Feb 08 2022 at 11:58):
I got alerted to this by yesterdays FHIR-I. Thanks for the write up @Josh Mandel . What is not clear to me is why the response needs to be wrapped it in a single-entry-bundle. Would it not be possible to return it unwrapped (possibly after a redirect)? It would avoid the bundle-in-bundle as well.
Josh Mandel (Feb 08 2022 at 15:17):
The motivation was to have a clear separation between the semantics of the response to the polling request and the semantics of the eventual response to the asynchronous operation. Bundle.entry.response is the one place in FHIR today where we explicitly model HTTP response headers and status codes, so I thought it made sense to reuse.
Last updated: Apr 12 2022 at 19:14 UTC
