Stream: implementers
Topic: FHIR Subscription Implementation
Mohinder Dick (Mar 09 2017 at 16:14):
Is anyone aware of an implementation to the FHIR subscription standard. I am interested in integrating notifications in my system and I want to avoid building subscription support from scratch.
Patrick Werner (Mar 09 2017 at 18:49):
HAPI is supporting subscriptions, but only with websockets atm
Mohinder Dick (Mar 09 2017 at 19:02):
Do you have any experience with it? I tried fining the functionality by digging around in the source and didn't see anything promising.
Mohinder Dick (Mar 09 2017 at 19:04):
@Patrick Werner Does it mean I can submit a subscription to the FHIR server and it will notify me when a resources matches my criteria.
nicola (RIO/SS) (Mar 09 2017 at 19:04):
We are working on reactive search by long polling - comming soon :)
Mohinder Dick (Mar 09 2017 at 19:09):
@nicola (RIO) Is we an open source project? Polling my not work ideally for my application. I work for a hospital network. We want to compare many subscriptions to a given FHIR resource and trigger when the ones that criteria that match.
nicola (RIO/SS) (Mar 09 2017 at 19:12):
No we are not open source, but i will describe implementation details in blog post :) Why do you think - polling not work for you? We've implemented a lot of reactive api's - and polling is most robust approach. What kind of criteria do you plan to subscribe - could it be expressed as search?
Patrick Werner (Mar 09 2017 at 20:09):
@Mohinder Dick here is some explanation how it works: http://build.fhir.org/subscription.html#12.25.6.2
You should be able to experiment with the public HAPI Testservers, or faster - with a local hapi CLI Instance
Abbie Watson (Mar 09 2017 at 20:31):
Clinical Meteor has been using a websocket based pub/sub protocol for about 3 years now; and we've been shipping FHIR resources across it for nearly a year now with the Meteor on FHIR project. Basically, we use ddp to set up the subscriptions and minimongo for the client side caching. The DDP subscriptions use an in-house JSON object, however, and while I've drafted out how to formalize things with the FHIR Subscription schema, I've yet to implement a canonical example. But it's all right there, and only requires an afternoon of work. I'm in the middle of finals week, so I don't have that afternoon right now; but if you can wait until the week of the 19th or 20th, I can provide a full implementation.
Abbie Watson (Mar 09 2017 at 20:47):
Here's a gist...
https://gist.github.com/awatson1978/e6f23432c500f14451013b7c20502695
Basically, a database query gets defined on the client and put into the criteria field, which is sent over to the server, where it's used to query the database and create a publication. In a non-Meteor application, the pub/sub methods are attached to a ddp object, I believe. DDP handles negotiating the URL endpoints, and upgrading to a websocket channel.
The underlying implementation works splendidly, and supports schema validation, cache-invalidation, rate throttling, limits, sorts, offline caching, resync, optimistic UI, etc. It's particularly nice having isomorphic schema validations on both server and client, checking that FHIR resources are properly shaped throughout the data layer.
Mohinder Dick (Mar 09 2017 at 21:45):
Hi @nicola (RIO), our scenario is event-driven. Messages are arriving in a stream. Polling requires you to read from a source at an interval and do the evaluation them. In our case, we need to evaluate as messages arrive from a source that is calling our system.
Mohinder Dick (Mar 09 2017 at 22:10):
Hi @Abigail Watson , Can you help me overcome two challenges with your suggestion? One is how to translate the criterion in the FHIR subscription into the mongo query. The FHIR subscription resources does not specify the criterion in the same format your example does. The criterion is a search string like this ``. I need to get that into a query to use your solution.
Two, How do I query a stream of data using mongo? In my scenario and external system is feeding me the FHIR researches to evaluate and send out within a reasonable time. I am not sure can store, retrieve, evaluate then send the notification with sufficiently low latency.
That said, I do find Clinical Meteor interesting. How do I experiment with it?
I will also check out your meteor on FHIR repo. Kudos.
Jens Villadsen (Mar 09 2017 at 22:37):
I've been puzzling with this myself. I would suggest a Server-sent events (even though its not part of the spec ... yet ) approach and use FhirPath for criteria.
Mohinder Dick (Mar 09 2017 at 22:42):
@Patrick Werner I know how the subscriptions work. I just don't think HAPI FHIR has the capability to actually send the notifications. I am investigating now.
Abbie Watson (Mar 09 2017 at 22:45):
I updated the gist so that the criteria only depends on a string. In practice, you could do all sorts of things with that field... send an SQL query, serialize an object via a URL, use FhirPath, etc.
As to your second question, it's not necessary to write to Mongo. There are Meteor implementations that are kept entirely in RAM and which use Redis. Mongo is just super convenient because it's a JSON store, and can act as a persistent caching layer to what's natively stored in the browser. No lookups/joins means it's very fast.
https://github.com/clinical-meteor/meteor-on-fhir
Mohinder Dick (Mar 09 2017 at 22:51):
Hi @Abigail Watson you are teaching me a lot. I didn't know that FHIRPath was a thing!!!
However, I still don't see how you translate the criterion string into a mongo query. Your example seems to assume the resource type is subscription. I need to support the FHIR criterion is stated as a FHIR search (see https://www.hl7.org/fhir/search.html). So it will look like this [base]/Patient?identifier=http://acme.org/patient|23. This example is asking for the patient resource type with an identifier of "http://acme.org/patient or an _id of 23.
The search spec has a lot of nuances like modifiers (e.g. ne, gt) and resource-global parameters (e.g._lastUpdated). They can also be stated in three different formats. My example is in the format [base]\[resource-type]/[parameters].
Abbie Watson (Mar 09 2017 at 23:04):
Like I said, it's not a completely finalized implementation (yet). Pretty sure we implemented the pub/sub infrastructure before the Subscription resource was even created. There's actually a whole lot more to the pub/sub infrastructure beyond the search criteria (websocket protocol negotiation/bootstraping, cache invalidation, rate throttling, offline, etc). So we've been stress testing and working on those features. But the search functionality is fairly straight forward problem of deserializing a string into a JSON object. Fairly trivial compared to getting cache invalidation right. Just need a free afternoon to finalize the implementation to spec.
Abbie Watson (Mar 09 2017 at 23:09):
Another update to the gist...
https://gist.github.com/awatson1978/e6f23432c500f14451013b7c20502695
Abbie Watson (Mar 09 2017 at 23:10):
As you can see, the approach is pretty malleable and flexible. Mostly a question of how much FHIR specific functionality to bake in by default. i.e. How do we want to handle malformed search requests, etc.
Mohinder Dick (Mar 09 2017 at 23:18):
Hi @Abigail Watson cool. I would be very interested in your implementation of parseCriteriaString.
Abbie Watson (Mar 09 2017 at 23:46):
There's now a minimal implementation of the parseCritieraString in the gist. Don't have time to do more today; but should get people started if they want to tinker with it. That's what a basic FHIR Subscription looks like using Meteor.js. :)
nicola (RIO/SS) (Mar 10 2017 at 05:46):
@Mohinder Dick long-polling is little bit different from just polling - https://en.wikipedia.org/wiki/Push_technology#Long_polling
Mohinder Dick (Mar 10 2017 at 12:23):
@nicola (RIO) I understand long-polling. It's just not appropriate for my scenario. In both polling and long polling the client, the application requesting the resource, can address the server and get a response. In our scenario, we do not assume the client can poll, because it may not have a reference to our system our system (the server). However, our system has a reference to the end-point of the client.
Jeff Chung (Mar 13 2017 at 22:45):
I have an implementation for rest-hook subscriptions that is planned to be added to HAPI.
Jeff Chung (Mar 13 2017 at 22:52):
I have a question though. I am going through the FHIR Subscription specification here: https://www.hl7.org/fhir/2017Jan/subscription.html
Is the client ever notified if someone deletes a resource that matches their subscription? This sentence leads me to believe subscriptions do not notify the client. "When a resource is created or updated that meets the criteria, the server sends a POST request with no body to the nominated URL." I don't see any documentation on what happens about deletions.
Jeff Chung (Mar 13 2017 at 22:55):
@Mohinder Dick I have an implementation that uses the interceptor framework so it does not use polling.
Mohinder Dick (Mar 14 2017 at 14:04):
Hi @Jeff Chung , I believe you are correct that you would not get a notification on delete.
John Moehrke (Mar 14 2017 at 15:00):
I think you should submit a CR to get this clarified. I suspect you are not alone in wanting notification on delete.
Jeff Chung (Mar 14 2017 at 15:09):
@John Moehrke That's a good idea. We may need a way to tell the client the type of change. Was the notification sent due to a create, update, or delete?
Jens Villadsen (Mar 14 2017 at 15:34):
Indeed ... a notification should be sent when it is deleted and previously would fall under the stated criteria. You got my vote to get that changed ASAP
Patrick Werner (Mar 15 2017 at 09:08):
@Jeff Chung good idea, it is absolutely neede to get notifications on deletion
Brian Postlethwaite (Mar 16 2017 at 06:33):
_history has deletes in the bundles, but searches with criteria don't handle deletes.
In this same bucket is when something that was in you filter is updated, and not it isn't should possibly come through as a notification, but not sure how you say that in the output bundle.
John Moehrke (Mar 16 2017 at 12:17):
I suspect that delete would be subscribed for differently. I will note that in IHE XDS there is the DSUB profile (Document Subscription). It registers a filter (similar to a query) that when new (create/update) information matches the filter, the notification is sent. Uses WS-BaseNotification. This also does not work for delete actions. There is interest to create a way to subscribe to delete, but it will likely be a different mechanism. It is more a request to listen to _history on an instance, not a filter/query match.
Grahame Grieve (Mar 16 2017 at 18:49):
subscription doesn't handle delete - that is inherent in the way it is designed
Mohinder Dick (Mar 27 2017 at 12:53):
@Jeff Chung Thanks for your help exploring the rest-hook implementation. I think it meets most of our requirements.
Patrick Werner (Apr 06 2017 at 11:38):
@Mohinder Dick we finally had the time to dig deeper into the hapi websocket implementation, websockets are not activated by default in the hapi-jpa-example
@James Agnew is there a reason why websockets are not enabled by default?
In order to get websocket support in Hapi you have to add the following to your FhirServerConfig(Dstu3) Class:
@Import({WebsocketDstu3Config.class, WebsocketDstu3DispatcherConfig.class})
then you should see something like this in the startup log:
“2017-04-05 01:42:56.048 [localhost-startStop-1] INFO o.s.w.s.s.s.WebSocketHandlerMapping [AbstractUrlHandlerMapping.java:354] Mapped URL path [/websocket/dstu3] onto handler of type [class org.springframework.web.socket.server.support.WebSocketHttpRequestHandler]”
Patrick Werner (Apr 06 2017 at 11:39):
we already created a java based client helper class for hapi websockets and are willing to contribute it to the hapi codebase, if there is interest in this
Last updated: Apr 12 2022 at 19:14 UTC
