Stream: implementers
Topic: Slow Loading Data Sources
Matthew Spielman (Sep 27 2017 at 14:09):
Outside the context of primary clinical systems (i.e. EHRs), it is common to have slow loading sources of data. This is particularly true with HIEs where data sources like eHealth Exchange or public health database queries can often take several seconds to respond. If that data is being aggregated and presented as FHIR to a client system/application, then the Server has two options:
1) Wait for all data to return (or time out) and then serve resources to client
or
2) Return immediately available data at query time and continue to background load slow data sources
Option 1 presents challenges in that it's violating the "F" in FHIR, and would potentially delay load times as we wait for the complete record to be available.
Option 2 is much faster because we can immediately respond with the data that is already on-hand. However, there is the challenge of how to indicate to the client that slower data sources are still loading and the record is incomplete.
There is nothing in the standard to accommodate something like Option 2 which means that if the application only fetches data one time, it will never get the new stuff and if it does refresh from the server periodically, new data may unexpectedly appear.
Has anyone implemented an operation or some other kind of flag to either indicate that data is still loading or that new data is available on the server for a particular patient?
Lloyd McKenzie (Sep 27 2017 at 14:57):
#2 seems like a UI issue, which is outside FHIR's scope. There are lots of UI cues you could use to indicate that data is still being retrieved - and hour glass, a slightly greyed-out screen, etc. The real trick is letting the user know they can interact with the data before the load is "complete". The other is a clinical safety issue which is "is it safe to let the user interact with the data before they have the full picture?" - and that you're going to have to seek guidance from your clinicians and your legal team...
Matthew Spielman (Sep 27 2017 at 14:59):
Well it's both a UI issue and a backend issue. The UI needs something to let it know that data is still loading. It's not a question of data being sent slowly to the client, it's a slow load to the server where it's stored in an ephemeral cache specific to the client "session".
Lloyd McKenzie (Sep 27 2017 at 15:03):
So the issue is that the data the client queries could be out-of-date a minute or so later? In that case, the client would either need to repeatedly poll based on lastUpdated or would need to set up a subscription.
Ewout Kramer (Sep 27 2017 at 15:42):
@Matthew Spielman , as part of the ONC's request to add new capabilities to FHIR to support bulk datasets, we are researching and proposing an asynchronous option for some/most/all FHIR operations and interactions. You can read more about this here: http://www.healthintersections.com.au/?p=2689
Lloyd McKenzie (Sep 27 2017 at 21:00):
That mechanism would allow for asynchronous processing, but wouldn't support continuous updating - except by polling.
Yunwei Wang (Sep 28 2017 at 13:24):
Or subscription. Client can register subscription for certain resources and server push notification when changes happens.
Oh never mind. @Lloyd McKenzie already mentioned that in one of his posts
Last updated: Apr 12 2022 at 19:14 UTC
