Stream: implementers
Topic: synchronous in front of asynchronous
David Hay (May 02 2016 at 03:52):
Suppose you have an architecture where the actual storage is asynchronous (e.g. via a queue) but you want to place a synchronous interface in front. The resource is validated by the synchronous end point and assigned an id before being placed on the queue for storage and returning success to the client.
The obvious problem is that if the client re-reads the resource too quickly – before the async backend has saved the resource - then it won’t be present.
This would certainly be confusing to the client, but is it an absolute prohibition? Ie that the server can only return success to the client after the resource has been persisted and is available for a subsequent read?
Brian Postlethwaite (May 02 2016 at 03:54):
I've been contemplating this one quite a bit and come to a happy medium for my server (not implemented yet though)
Brian Postlethwaite (May 02 2016 at 03:56):
I'm going to persist the resource, and id and then indexes will update later.
Brian Postlethwaite (May 02 2016 at 03:57):
Maybe 1 or 2 core indexes might update in real time, but most will be async.
David Hay (May 02 2016 at 04:15):
but, if a client were to retrieve the resource directly by the id, the resource would be returned - right?
Grahame Grieve (May 02 2016 at 06:50):
The same problem occurs in a threaded synchronous architecture too, where someone asks to read the resource before the response to an update has been processed
Grahame Grieve (May 02 2016 at 06:50):
this is a classical problem in information systems; Martin Fowler has written books about it
David Hay (May 02 2016 at 07:06):
Does RESTful FHIR have a position on this? ie does returning 201 (created) mean that the resource is saved and ready for retrieval? The w3 spec states that a 201 response code means "The request has been fulfilled and resulted in a new resource being created" and "The origin server MUST create the resource before returning the 201 status code. If the action cannot be carried out immediately, the server SHOULD respond with 202 (Accepted) response instead." - suggesting that when the persistent storage is queue based, either the response should wait until confirmation that the resource has been saved (and therefore ready for query), or immediately return a 202 which we don't currently mention...
Grahame Grieve (May 02 2016 at 07:07):
we don't support 202, right. Complexity... but w3c don't make rules about parallel requests
David Hay (May 02 2016 at 07:14):
No, but they do make rules about the behaviour of HTTP - is there a case for supporting 202? For that matter, does the fact that it is not explicitly mentioned in the spec mean that a queue based system shouldn't use 202 rather than 201?
Grahame Grieve (May 02 2016 at 07:49):
but that makes sense - how could you know in advance that you were guaranteed to accept something, without actually accepting it?
Grahame Grieve (May 02 2016 at 07:49):
so a queue based system has to wait for the internal response before responding to the client, since we don't support 202
Grahame Grieve (May 02 2016 at 07:49):
202 is only good for dumb content
Michael Lawley (May 06 2016 at 18:53):
Why do you say that? 202 only means the request has been accepted (in this case, successfully added to the queue) It doesn't imply that it will be successfully processed
Grahame Grieve (May 06 2016 at 20:42):
what happens if it fails? how does the client find out, and why?
Michael Lawley (May 06 2016 at 21:10):
Generally the 202 response includes a link (Location header) to some kind of status resource (following HATEOAS)
This is a handy summary of the approach http://farazdagi.com/blog/2014/rest-long-running-jobs/
Grahame Grieve (May 06 2016 at 21:12):
as usual, HATEOS means badly thought out ad-hocness.
Grahame Grieve (May 06 2016 at 21:13):
It's possible we could add this, but it's a fair bit of machinery to make it interoperable
Michael Lawley (May 06 2016 at 21:23):
Would you need more than a way to represent "in-progress" in an OperationOutcome? Interop wouldn't need you to standardise the Location URL
James Agnew (May 06 2016 at 21:25):
Why does any of this need to be standardized? This sounds like an implementation detail for a specific scenario...
Grahame Grieve (May 06 2016 at 21:25):
success, failure, reason
Grahame Grieve (May 06 2016 at 21:26):
well, depends on whether you want it to be interoperable or not
Grahame Grieve (May 06 2016 at 21:27):
you certainly wouldn't expect any client to understand that a 202 + a location tells you the location a task, not the outcome. And you'd need to inject the location of the outcome into the task response somehow as well. So you could do this, but it would be non-conformant as things stand
James Agnew (May 06 2016 at 21:30):
I guess this is like those two phase accept ACKs you could do in HL7v2 (whose name I'm drawing a blank on right now). I always heard that there were people who actually used this, but I have never witnessed it.
Do people actually expect an interoperable solution to this problem? Do other APIs of any kind have one? I just fret a bit that we're layering a lot of infrastructure on top of FHIR these days.
Michael Lawley (May 06 2016 at 21:36):
yes, you'd need to standardise this approach to a 202, but I don't see it as a lot:
202 + task Location which, when resolved, then gives you either:
a) OperationOutcome indicating that its still in progress
b) OperationOutcome indicating the failure (or success)
c) redirect to the success Resource
d) 404 when the task has been garbage collected
Lloyd McKenzie (May 06 2016 at 23:29):
And how many of those with this need can't do messaging? (Messaging is designed for asynchronous - REST really isn't.)
Michael Lawley (May 07 2016 at 05:41):
My understanding of Messaging is that the server needs to send the response nack (to MessageHeader.source.endpoint], but in this use-case there is no endpoint to send responses to
Lloyd McKenzie (May 07 2016 at 11:36):
Message delivery can be done by putting things in a "mailbox" and having the client poll for new content. We haven't defined an explicit mechanism for this with FHIR yet, but we had it with v3. And it might be more appealing than adding complexity to REST.
Last updated: Apr 12 2022 at 19:14 UTC
