Stream: subscriptions
Topic: Response to rest-subscription with payload
Allan Bro Hansen (Nov 14 2017 at 15:50):
When a system POST’s a resource (the payload) to a subscriber (the channel.endpoint):
Do we have any guidelines in respect to the response given by the subscriber?
- Should the subscriber only be allowed to validate the format (given by the Structuredefintion) of the resource?
- Or is it perfectly accepted and expected if the subscriber also chooses to perform its own business validation – corresponding to the description of the response to $process-message https://www.hl7.org/fhir/messaging.html#process
Grahame Grieve (Nov 14 2017 at 15:53):
well, the real question is, what happens if it returns an error. Off the top of my head, I don't know what my system would do
Allan Bro Hansen (Nov 14 2017 at 16:00):
I guess my system would log this at an error, and hence notify Helpdesk. This could e.g. be the case if the auth-header had a invalid bearer-token which could be fixed by updating the subscription.
Allan Bro Hansen (Nov 14 2017 at 16:04):
So the question is if the notifying system is expected to handle more than just configuration and set-up errors? (I would also handle an invalid url and its resulting time-out)
Nick Hatt (Nov 14 2017 at 16:54):
@Allan Bro Hansen we deal with this at Redox a lot - even to our own internal FHIR server (@Benjamin Flessner I'm looking at you), but we also send to hundreds of applications that have totally different expectations for errors, response codes, and validation.
A few good principles:
# Don't trust that the receiver will ever do anything you want them to. It would be great if we could force people to send a Retry-After header if they know they are down and coming back up, but it's not realistic.
# If you bake in way to do retries, bake in a circuit break too https://martinfowler.com/bliki/CircuitBreaker.html - the reason the subscriber is sending 500s might be because you are breaking them.
# What you do with 400-ish response really should be up to you and your organization. It's much more of a cultural and value-oriented decision than a technical one. If someone can't get a subscription push at 4am, are you going to page someone?
We've sort of had to build a lot of this infrastructure through trial and error and trying to scale a subscription model. That's why I think the value part of it is huge in how you design your system. Our applications are paying us to make sure they keep getting push messages, so we wake people up in the middle of the night, even if it's something we can't fix.
Allan Bro Hansen (Nov 14 2017 at 19:23):
@Nick Hatt: Thank for the inputs, and your response.
My question was however not how I should handle errors but rather if it would be expected FHIR-Subscription behavior that the subscriber performs business validations. I understand that I cannot force any behavior upon the subscribers but I can certainly choose to expect that they behave as agreed.
My starting point was a rest-subscription without POSTing any payload making it a pure notification about “your subscriber-event/topic has happened” – the subscriber could then perform a query getting details about what did happen, and choose to act upon this. Without the payload, the subscriber would not be able to perform business validations as part of the GET and say that they did not like/accept what did happen. This would call for some different kind of “complain” interaction, if any.
I now have chosen to include the payload and hence giving the subscriber the possibility to validate the “what has happen”-part. This seems a bit asymmetrically compared to the no-payload scenario but it could make sense . A use case could be “we want your system to monitor the complete subscriber notification and transaction at the subscriber part” (and call people at 4 am if it fails).
I find very little of expected subscriber-response at https://www.hl7.org/fhir/subscription.html. Hence my question.
Nick Hatt (Nov 14 2017 at 20:28):
Isn't a rest-hook just a FHIR update interaction https://www.hl7.org/fhir/http.html#update?, so an OperationOutcome https://www.hl7.org/fhir/operationoutcome.html would be appropriate.
Allan Bro Hansen (Nov 14 2017 at 20:47):
yes - I guess you are right. It says: "This requests that a server forward a copy of any matching resource in JSON format to the nominated server as an Update operation using the nominated URL as the service base. ".
So in fact I guess it is a PUT with the normal validation handling via operationOutcome.
(although I am really not asking the subscriber to update anything)
Allan Bro Hansen (Nov 14 2017 at 20:47):
Thanks :-)
Jens Villadsen (Nov 14 2017 at 23:10):
I recently did a 'field-study' on the FHIR validators out there and they do not all entirely agree upon everything - which means that there is no such thing as 'normal validation' - that is at least not my experience. If you are at DevDays @Allan Bro Hansen come and join the discussion we are gonna have on the subscription resource. I'm currently authoring a gist with some findings I would like to talk about.
Grahame Grieve (Nov 15 2017 at 04:28):
of course we want convergence from the validators
Grahame Grieve (Nov 15 2017 at 04:29):
maybe we should document use of the Retry-after header in the subscription rest-hook doco - I didn't know about it
Allan Bro Hansen (Nov 15 2017 at 07:59):
@Jens Villadsen : Sadly I am not in Amsterdam. I am a bit surprised by the implications of returning a payload in stead of omitting it. Naively I would have expected just to add the payload as body (still doing a POST to the channel.endpoint)
Jens Villadsen (Nov 15 2017 at 07:59):
@Allan Bro Hansen - if you any stuff you would like me to bring up besides https://gist.github.com/jkiddo/e773d033c245c2c4fdef6d35cbe1a75b let me know
Jens Villadsen (Nov 15 2017 at 08:02):
But yeah - it takes some balls to use payload. It also comes with a huge performance gain ... and a lot of complexity pain
Grahame Grieve (Nov 15 2017 at 08:51):
y
Grahame Grieve (Nov 15 2017 at 08:51):
but yes. just add the payload as a body
Allan Bro Hansen (Nov 22 2017 at 07:53):
My point was that it is not "just add the payload as a body" - it is to perform a PUT a different url (channel.endpoint/<ressourceName>/<ressourceId>) - and maybe also implicitly to allow be the subscriber to perform business rules.
Jens Villadsen (Nov 22 2017 at 07:57):
And now the tricky part comes into play: What if the business logic of the 3. party rejects your update? I guess you should run a retry if it is in the 5xx range, but what if you hit a 4xx. Should you retry? How often should you retry? Should you retry at all?
Grahame Grieve (Nov 22 2017 at 08:04):
Some of these questions are taken up in the W3C subscriptions spec. @Kevin Marks is going catalogue the differences and we'll copy anything we can
Jens Villadsen (Nov 22 2017 at 08:17):
Should I wait for @Kevin Marks to return with some findings before I start making change requests on gforge from https://gist.github.com/jkiddo/e773d033c245c2c4fdef6d35cbe1a75b ?
Last updated: Apr 12 2022 at 19:14 UTC
