FHIR Chat · references · subscriptions

Its been a while since I've been into FHIR subscriptions so I would like to post a question regarding some observations. First a bit of background context:
Lets first differentiate between "ad hoc subscriptions" and "system subscriptions". I consider "ad hoc subscriptions" as subscriptions that are created ad hoc by a system or a user for a brief moment in time that has a "short lifespan" - probably scoped by the duration of a session a user is logged in/interested in events. These are not the subscriptions that are in particular focus of my upcoming question. "system subscriptions" on the other hand, I define as subscriptions that live as long as the system is running. The nature of "system subscriptions" is that they typically have a broader query string and are less context aware (as the query string is less specific). Due to security reasons, I also believe that "system subscriptions" would probably have a tendency to be-id-only as payload as more events will be sent.

The question:
Now, if "system subscriptions" are used with id-only and the primary reason for using "system subscriptions" is optimisation, wouldn't it be beneficial if besides sending the id, that also reference id's could be present (since my thought is that this is done for optimisation).

Let's imagine a client (with no persistent state) to a system with a "system subscription" that subscribes on Observations being created and/or updated. If events from this system should be really usable and secure, the full observation is not sent - meaning the payload type is id-only. But since the client gets a notification on a observation it more or less should query the system for the details on that observation in order to determine whether the observation is relevant for the client.

Lets say there are two patients (A and B) stored in the system and the client is currently only working on patient A (remember - the client has no persistence but may have a cache on current resources) - and that the observation that it received an event for regards patient B - then there is no reason for the client to go and fetch the observation as it is linked to patient B. What if the subscription could be set up to also contain a configurable set of references in its events so that events on the "system subscriptions" would still be id-only but also contain eg. (configurable) Observation.subject. With that information, the client would quickly be able to know that the event just received would concern an id to a patient that is currently not of particular interest of its user? Wouldn't that be of use - eg. a new category type that is not id-only and not full resource - but "id-only-with-references"?

This is a tradeoff of course. Instead of using "system subscription" the client could instead use "ad hoc subscriptions" - with the penalty of being the bookkeeping of creating and deleting subscriptions. Also using "system subscription" and "id-only-with-references" requires the system to emit events to clients that may never be of interest.

But am I on to something, has this case already been discussed, is "id-only-with-references" merely a subscription with a proper _elements query and with type full-resource, or am I dreaming up an exotic case?

Vassil Peytchev (Apr 30 2020 at 12:36):

Maybe take a look at the reworked (and still in progress) subscription framework? In particular, the ability to specify a FhirPath trigger in SubscriptionTopic can allow the client to describe that observations for patient A are the only ones needed...

Jens Villadsen (Apr 30 2020 at 13:22):

Then it would be back to "ad hoc subscriptions". I would like the trigger to stay the same across clients.

Vassil Peytchev (Apr 30 2020 at 13:33):

The new Notification bundle can handle the notification you are looking for. The question is, can you get a SubscriptionTopic or Subscription to describe that type of notification.

Gino Canessa (Apr 30 2020 at 15:52):

Hi Jens, it's a long scenario, so I've broken up my thoughts/comments below. Note that everything I am referring to is from the R5 branch (new model).

There has been discussion around graphs of both resources and IDs (what you are describing).
- The primary reason it has been tabled is security. Specifically, trying to follow references or build graphs without an active security context (since subscriptions are not tied to a logged-in user) is challenging at best. Even in the cases where the is a user logged in, the normal pipeline is bypassed (since the processing is not in that context), so vendors would need to re-implement their security and maintain it twice.
- Overall, people like the idea of graphs - but we can't afford to hold up the core functionality to try figure out how to do it.
That said, I am a little confused on the scenario. A Subscription includes endpoint information, so multiple clients cannot reasonably share a single Subscription. Multiple Subscriptions can point to the same SubscriptionTopic, which means they would have the same core definition, but could each apply their own filters as well. The way I would set up what you described would be:
- A SubscriptionTopic, which defines the foundation of what you want (e.g., create or update on Observation). I would suggest filtering on Observation.patient instead of Observation.subject in this scenario, since filtering uses search parameters and it fits the situation better.
- A client interested in notifications for a patient needs to create a subscription for itself (the client), since otherwise the server has nowhere to send notifications to. The Subscription contains:
  - endpoint information (where to send notifications)
  - a filter for the patient of interest
  - a relatively soon end so that the server will automatically clean up the Subscription.
- Since notification bundles include the Subscription reference, the client implicitly knows which patient a notification is for (assuming there is some way to link them).
  - If the client has NO ability to track which subscriptions it asked for (I'm not sure how that's possible), you could embed needed information into the endpoint configuration of the Subscription. Be very careful here to not include PHI!!! E.g., the endpoint could be .../notifications/facilityXYZ/patientA, or a header could be specified: subscription-reference-information=patientA.

In the new model, the trigger is defined in the SubscriptionTopic, while the Subscription defines endpoint information, (allowed) filters, etc. This means that the same trigger can be used for every client, but with filters to differentiate between instances.

Does that help/make sense?

Keith Boone (May 01 2020 at 07:31):

@David Pyke The conversation above probably covers what you were asking for a few days ago...

Jens Villadsen (May 01 2020 at 19:27):

@Gino Canessa - well, parts of it helps. First - I haven't been digging into the new model on Subscriptions yet so please bear with me. You say that multiple clients cannot share a single subscription. I guess that depends on how you define clients. If my setup is as follows: a web frontend working on a set of horizontal scaleable BFFs (with a shared distributed cache) that interfaces the FHIR backend, and it is the BFF that creates a subscription on-demand then I would say that the BFFs shares the subscription, wouldn't you?

Gino Canessa (May 01 2020 at 20:45):

@Jens Villadsen , no worries! The updates to subscription aren't finished, so I assume most people aren't overly familiar.

From the subscription side, I consider the endpoint a single system. Whether it is a single process, a cluster of servers, or even multiple different systems doesn't matter. If I receive a text message, my phone number represents a single client, even if that message is broadcast to multiple devices. The originator neither knows nor cares what happens beyond initial delivery. So, from the FHIR server perspective, an endpoint is a client.

Going back to your scenario, a back-end can operate as a distribution point (e.g., rebroadcasting or redirecting notifications). Briefly, I can see a couple of models:

The back-end is a durable endpoint with long-lived subscriptions. Adding/removing patients of interest is managed, and notifications are always being received. The server decides which notifications are sent to which clients.
- This could be individual subscriptions for each patient, which can be filtered as described in my previous post.
- This could be a single subscription (e.g., patients in a group/list).
  - If the back-end service is more aware than just a web server, it can resolve notifications to get the Observation (containing the patient reference) and go from there. This prevents the front-end clients from needing to do anything, but adds more to the back-end.
Front-end clients dynamically add/remove subscriptions, using the back-end as a forwarding endpoint.
- My reference implementation works that way (note that it is a revision behind on models - updates are WIP). The server creates URLs for clients, and only sends notifications to the requesting UI (via websockets if you are curious).
  - The FHIR server doesn't ever actually know about the front-end clients. It just does a POST to the endpoint and moves on.
- If each subscription is for a patient, the client knows which subscription relates to which patient and can easily self-filter.
- If the client is using a group/list for subscriptions, then the client has to resolve the reference to figure out the patient.

I just saw something I missed in your initial post, which should help clarify as well. id-only payloads are the compromise between security and optimization. The optimal starting place for the client is a full-resource payload, since it gives many clients all the information they need. But, similar to the issues with graphs I've discussed, we can't have PHI floating around the internet. The 'most secure' way of doing that is to just ping a client that something happened (empty payload). But clients then need to query servers to figure out what happened, so the notification isn't much better than simple polling. By sending the ID of a single resource, there is very little risk to the FHIR server and minimal risk for relays (e.g., a server converting from HTTP POST to a platform-specific push notification), while not making clients constantly run expensive queries.

I'm happy to discuss any thoughts for how to improve, but I'm currently out of ideas - every path I go down has the potential for PHI in the message and will require a lot more discussion.

Last updated: Apr 12 2022 at 19:14 UTC

Main menu