Stream: implementers
Topic: Prefer Header
Grahame Grieve (Apr 16 2016 at 01:51):
rom an implementer: I ran into different behaviors when creating a resource on different FHIR servers that brought up an interesting question. The issue was the following: I was creating a new resource and expecting the full resource back in the response with the ID of the resource populated. Two servers did this by default. One (HAPI) did not. In order to provide a hint to the server to return the full resource rather than just the ID, we looked at the specification, in particular the ‘prefer’ header. The specification states that servers can choose either behavior as the default but that callers can specify which one the prefer using the prefer header. That part is great. However, it then states that server implementors SHOULD honor this. This brings us back to square one in the sense that one still is not sure what one is going to get. We thought that if this is not a MUST, then at least adherence to prefer should be specified in the conformance statement in some way. Yet, we could not find where one could find this in the conformance statement.
Josh Mandel (Apr 16 2016 at 03:29):
It's a fair point. Can someone remind me why respecting the prefer header isn't a MUST?
Grahame Grieve (Apr 16 2016 at 03:36):
I can't find the record
Peter Scholz (Apr 16 2016 at 06:43):
I think that it is implied by RFC 7240:
"The Prefer request header field is used to indicate that particular server behaviors are preferred by the client but are not required for successful completion of the request."
Kevin Mayfield (Apr 16 2016 at 07:43):
I would say it's a MUST. On most interfaces we have, we're only using the http responseCode and/or full resource. Also normally develop without operationOutcome responses as errors are more useful in original format (except for 400-499 errors).
James Agnew (Apr 18 2016 at 16:59):
I'm curious why so many people seem to want servers to return the resource that they just sent it. It doesn't seem useful to me, but obviously lots of people disagree.
Andy Stechishin (Apr 18 2016 at 17:04):
@James Agnew I think it may be because many of the RESTful frameworks do this as default behaviour. People simply have an expectation that this is 'what you do'.
Lloyd McKenzie (Apr 18 2016 at 17:10):
As well, servers aren't required to store exactly what they're given, so if you want to do a subsequent update, passing back what was actually stored saves you from having to do a GET before submitting an update.
Claude Nanjo (Apr 18 2016 at 17:48):
And James, in this case, whatever is the default behavior by the server, the ask is that it be predictable somehow. In the current spec, it is unpredictable.
Paul Knapp (Apr 18 2016 at 17:53):
I thought we had determined the default to be to not return the resource as this conserved bandwidth.
Grahame Grieve (Apr 18 2016 at 18:05):
no. we said that the default behavior was up to the server since this allows them to decide what is optimal in their context. the prefer header is used to make the behaviour predictable
James Agnew (Apr 18 2016 at 19:20):
We could always say something to the effect of "Servers SHALL return the updated resource content if they do not support the Prefer header", which would help on the predictability front
Brian Postlethwaite (Apr 18 2016 at 20:27):
That seems like a reasonable suggestion @James Agnew
Brian Postlethwaite (Apr 18 2016 at 20:28):
But then there is no place to "register" that the server supports the prefer header? (nothing in conformance)
Brian Postlethwaite (Apr 18 2016 at 20:29):
Maybe this needs to be a part of the patient track at connectathons to get people aware that this can happen.
Paul Knapp (Apr 18 2016 at 20:30):
If we are going to allow Servers to do just one choice of send or not send resource then I'd rather the not send is the default. Alternately if severs are required to support both then clients can set their expected behaviour by using the Prefer header.
Vadim Peretokin (Apr 18 2016 at 22:49):
I find it useful to read the new id from it, for when I don't have access to the response headers.
Grahame Grieve (Apr 18 2016 at 23:11):
when would you not have access to the response headers?
Vadim Peretokin (Apr 19 2016 at 00:47):
XSLT? Although I'm not doing this in XSLT. I'll stop being lazy and add the Prefer header.
Paul Knapp (Apr 19 2016 at 06:40):
In order to for you to be assured to get the 'Prefered' behaviour we would need to require servers to follow the Prefer header whereas the current behaviour is might follow.
Grahame Grieve (Apr 19 2016 at 06:42):
right. I'm ambivalent about this. From a client's point of view, it would be good to have that the server SHALL observe the prefer header. But it seems to me that there's lots of times I'm writing a server where I know which I'm going to do, and forcing the server to check the prefer header is overhead, particularly since it's about optimization
Paul Knapp (Apr 19 2016 at 06:43):
In Vadim's case if he provides the Prefer header and the server is allowed to ignore then he is 'hooped'.
Grahame Grieve (Apr 19 2016 at 07:50):
well, if the server doesn't return the content, yes
Peter Scholz (Apr 19 2016 at 16:16):
Life is no game of wishful thinking,
The RFC defining the Prefer Header does state that it is solely at the discretion of the server what amount it returns. The prefer return header allows the server to take into consideration what the client wishes. That's exactly how it is defined in FHIR when it reads a sever SHOULD honor the header. A MUST definition is contrary to the RFC
Ewout Kramer (Apr 20 2016 at 08:56):
Otherwise, I guess the header would have been called "Require" rather than "Prefer" I guess
Paul Knapp (Apr 21 2016 at 08:42):
So then this will work just fine in all cases except for when it doesn't - does that provide the needed level of consistency and assurance?
Peter Scholz (Apr 21 2016 at 09:42):
It should work fine for everyone, except for those clients making false assumptions
If you do ask for a minimal return, you can ignore the body if the server does not honor the Prefer return header.
If you need the resource persisted, and the server does not honor the return=represenstion, you can take the ID from the header an perform a read.
So where's the point ?
Paul Knapp (Apr 21 2016 at 10:45):
Functionally, as long as those who need the Id can obtain it correctly from the location header then their need will be satisfied.
You are correct, the the return of unncessary content, or the lack of supression of that through ignoring of the Prefered header, does not present a fundamentatl problem, it is merely a waste. Some will care, others will not.
Peter Scholz (Apr 21 2016 at 10:50):
It's like everywhere else,
those who follow the specs will have no problem, those who don't might fail
And the spec for Prefer headers is RFC 7240
Paul Knapp (Apr 21 2016 at 10:53):
That's true. It is unfortunate the spec wasn't written to effectively manage the waste, but it is what it is. Ewout is also correct and maybe a Required header would better meet the need.
Peter Scholz (Apr 21 2016 at 11:01):
IMHO there is no need,
everyone implementing his client correctly will be able to gain the amount of information he needs.
Most servers will honor the Prefer return, as they don't want to waste bandwidth and performance
It's the same as the Accept-Encoding Header indicating if client might support gzip compression on content
Some servers will honor it and send compressed content, others won't. Thats the way HHTP works well for quite a while now.
we should not try to take all burden away from the clients, and put it on the shoulder of the servers.
Grahame Grieve (Apr 21 2016 at 12:45):
if w3c had defined a required header, we would have used it.
Elliot Silver (Apr 21 2016 at 17:08):
So, I admit this isn't my area of strength, but I've been following this conversation, and don't understand why you don't want to make it a requirement the server SHALL honor the header? I realize that this is stronger than the RFC, but we layer more stringent requirements on underlying standards in other places.
Also, although a client can deal with the different server behaviors, it adds complexity--the client needs be prepared to try two different approaches. First, I'd rather put complexity on the server, than the client. Secondly, if most servers have one behavior, many client implementers might not even recognize they should be prepared to encounter the other behavior.
Ewout Kramer (Apr 26 2016 at 18:07):
If you do ask for a minimal return, you can ignore the body if the server does not honor the Prefer return header.
If you need the resource persisted, and the server does not honor the return=represenstion, you can take the ID from the header an perform a read.
This is actually what the .NET API does do hide these differences for the client: when return=representation and the server does not obey the prefer header, the .NET API will send out a GET, and return the representation to the client after all.
James Agnew (Apr 29 2016 at 12:48):
That's a good idea!
I'm stealing it. :)
Ewout Kramer (May 02 2016 at 19:34):
Technically, you're not stealing this, since this was the outcome of the discussion between me, Grahame and yourself in Paris ;-) You've just forgotten about it ;-)
Brian Postlethwaite (May 02 2016 at 23:10):
And this conversation has reminded me to implement it in my server (done yesterday)
James Agnew (Jun 24 2016 at 02:11):
If the client sends Prefer: return=minimal, is the server allowed to return an OperationOutcome? This has always been my understanding, but as Lloyd points out the spec seems to indicate that the options are either the created/update resource body or no body at all (see here )
Brian Postlethwaite (Jun 24 2016 at 02:20):
Client's need to be able to handle the OperationOutcome case anyway for when the validation/save fails, and the results are returned.
Lloyd McKenzie (Jun 24 2016 at 02:52):
The question is: Can a Create return an OperationOutcome on a success?
Lloyd McKenzie (Jun 24 2016 at 02:52):
My understanding is that on a success, you either get back nothing or a copy of the resource that was created.
Brian Postlethwaite (Jun 24 2016 at 02:59):
Could you pass back warnings to the client using this mechanism?
Brian Postlethwaite (Jun 24 2016 at 03:00):
Errors cause a fail, warnings are happy for the save to complete, but do want to inform that something happened.
Lloyd McKenzie (Jun 24 2016 at 03:00):
Yes, that would be the purpose.
Brian Postlethwaite (Jun 24 2016 at 03:03):
Anyone logged the tracker yet to suggest this change?
(which some of us appear to have thought was the case anyway)
Lloyd McKenzie (Jun 24 2016 at 03:05):
Well, the problem is that there's sometimes a need to echo back the created resource (because you want to see what actually got stored - often not exactly what you sent). And you can't do both a copy of the created resource and an OperationOutcome, even though both might sometimes be relevant.
Grahame Grieve (Jun 24 2016 at 03:08):
I say that if you think there's utility in returning hints and warnings (seems like a big waste to me) you need it whether you want to hte body of the resource or not - that's an orthogonal issue.
Grahame Grieve (Jun 24 2016 at 03:09):
so the alternative is a X-FHIR-Notes header, that has an operation outcome in json
Lloyd McKenzie (Jun 24 2016 at 03:10):
Technically, we could put two resources in the Body too, but I think that would be less expected.
Lloyd McKenzie (Jun 24 2016 at 03:10):
But I'm correct that right now, OperationOutcome is prohibited as a response to a successful create, correct? (Because that's not what HAPI does right now.)
Grahame Grieve (Jun 24 2016 at 03:19):
right. that's not conformant
James Agnew (Jun 24 2016 at 09:46):
The idea of putting an OperationOutcome in a header is a non-starter unfortunately.
I experimented with this for a little bit, every create/update would get a header with an OO containing the validation results. As I quickly discovered, several pretty fundamental bits of web infrastructure (apache server, squid proxy, probably others) don't like header values longer than 1024 characters without special configuration. This is just asking for trouble.
Using a repeating header with a note in each repetition (i.e. not encoded in an OO) could be an option. Is there any reason not to allow an OO back though as HAPI is (currently incorrectly) doing? I think this behaviour should be allowable.. will file a tracker unless there is objection.
Grahame Grieve (Jun 24 2016 at 10:42):
well, the problem is that we have 2 difference choices: OO with warnings or not, and resource or not. And we have one on|off header
James Agnew (Jun 24 2016 at 12:22):
Yeah, agree this isn't perfect. But would allowing OO on response make things worse? I don't actually see any rationale for not allowing it.
Lloyd McKenzie (Jun 24 2016 at 15:14):
Well, if we have a usecase for OO on response, then we have a use-case for both OO and payload on response - so the question is, is there any way we can pass back both in a manner that won't be totally surprising/breaking?
James Agnew (Jun 24 2016 at 17:23):
I don't see any way that is nice.. You could
- Return a bundle with two entries (blech)
- Use an operation and return the two things in a Parameters (only slightly less blech)
Nonetheless, I don't think that means we shouldn't allow the OO back by itself. Tracker filed.
Elliot Silver (Jun 24 2016 at 18:44):
I'm sure FHIR already has a position on this, but what about a multi-part response?
Lloyd McKenzie (Jun 24 2016 at 19:34):
@Elliot Silver I was thinking the same - is there a prohibition against having multiple payloads in the body (without going to Bundle)?
James Agnew (Jun 24 2016 at 20:00):
It seems to me that would accomplish the same thing as returning a bundle, but in a perhaps less client-friendly way.
Grahame Grieve (Jun 24 2016 at 20:01):
yes much less friendly
Grahame Grieve (Jun 24 2016 at 20:03):
I don't actually see a usecase for returning the OO in this topic
James Agnew (Jun 24 2016 at 20:03):
I like to use it to return validation results
Lloyd McKenzie (Jun 24 2016 at 20:33):
The other use-case is to warn about data that you've ignored or changed from what was submitted.
Lloyd McKenzie (Jun 24 2016 at 20:34):
If you echo back the content, they can see what you did, but they don't have any information as to "why?".
Grahame Grieve (Jun 24 2016 at 22:13):
neither of these are use cases. What does the client do with it other than ignore it?
James Agnew (Jun 24 2016 at 22:16):
In our case, they just log it. Obviously not a clinical use case if that's the bar, but we find it quite useful for debugging.
Grahame Grieve (Jun 25 2016 at 08:35):
really belongs in the header then. According to the http standards mailing list, headers over 8k in length are common, so I'm surprised to heara it's a problem
James Agnew (Jun 27 2016 at 17:01):
That surprises me.
I looked it up, and the max length in Tomcat is 8k (not 1k as I remembered) oddly enough. According to this link Apache is also 8k, IIS 6 is 16k, and NGINx is 4k or 8k depending on platform.
Obviously that gives a fair bit of room, but the validator certainly managed to exceed the 8k limit when I had it running for people's casual testing.
I dunno, I'd feel uncomfortable recommending a solution that requires reconfiguring something as basic as Apache. There are lots of situations where this type of configuration isn't accessible to you, and this isn't something that fails gracefully.
Grahame Grieve (Jun 27 2016 at 20:49):
well, all that makes sense. What this comes to is: there's no un-surprising way to return both a resource and hints and warnings about it. And there's no way to manage a split two-way choice which is what we logically have here either
Lloyd McKenzie (Jun 27 2016 at 21:00):
Well, given that the OperationOutcome isn't persisted, if there's a need for both to come back, the logical thing would be to return the OperationOutcome and let the client query for the created resource if they need that too. But that behavior would be "surprising" from a RESTful perspective to most systems.
James Agnew (Jun 27 2016 at 21:08):
I dunno, I guess I'm in the minority here but I just don't find that behaviour surprising at all. Basically every (non-FHIR) REST API we have here at UHN follows the same pattern for POSTing data: You send it data in whatever structure it requires, and it replies with a much smaller and entirely different structure that contains any relevant status information about what it did.
To me, returning the same entity that it just received is the surprising behaviour. I get that there are valid reasons to do this, but banning returning a status object is just odd to me.
Elliot Silver (Jun 27 2016 at 21:13):
What about a header on the request that identifies whether the client wants back the saved resource (default), the OperationOutcome, or (in a multipart response) both? This gives the "usual" RESTful behaviour*, but also gives the option for info that a more advanced client can make use of.
(*I'm going to go with that spelling, given James, Lloyd, and Grahame's participation on the thread.)
James Agnew (Jun 27 2016 at 21:18):
oh man if we have corrupted an American into spelling behaviourcorrectly, this whole FHIR adventure has been worth it.
Elliot Silver (Jun 27 2016 at 21:21):
Sorry @James Agnew , not to invalidate your experience, but as a Canadian, I've always spelt it correctly.
James Agnew (Jun 27 2016 at 21:25):
lol ok I thought you were.... but your comment made me think otherwise :p
Grahame Grieve (Jun 27 2016 at 21:30):
well, the prefer header is more than just minimal vs representation. it's extensible, and servers are requried to ignore values they do not understand.
Grahame Grieve (Jun 27 2016 at 21:30):
so we could define a value 'warnings' to indicate that the client wishes for the server to return an OperationOutcome instead of the full resource
James Agnew (Jun 27 2016 at 21:31):
+1
Brian Postlethwaite (Jun 28 2016 at 00:59):
Something like this in the prefer header?
return=minimal
return=representation
return=outcome
Grahame Grieve (Jun 28 2016 at 01:01):
well, suggested:
return=minimal
return=representation
return=warnings
Grahame Grieve (Jun 28 2016 at 01:01):
but the exact word is not important
Brian Postlethwaite (Jun 28 2016 at 01:14):
warnings wouldn't be clear that you were always after the OperationOutcome, regardless of the result.
(which is how I'd interpretted what this discussion was after, and would be happy with)
Grahame Grieve (Jun 28 2016 at 01:15):
outcome isn't obvious that you don't want the outcome of the posting the resource ;-)
James Agnew (Jun 28 2016 at 01:16):
make it explicit
Prefer: return=operationoutcome
Brian Postlethwaite (Jun 28 2016 at 01:21):
:) This one.
Pascal Pfiffner (Jun 28 2016 at 11:24):
That last suggestion would be okay I guess. Contrary to James I'd totally expect the same resource back after a create/update from a RESTful service, never something entirely different (except on 400+).
Grahame Grieve (Jun 28 2016 at 19:38):
time for a task. GF#10250
Brian Postlethwaite (Mar 19 2019 at 00:38):
The spec is quiet about how to represent multiple prefer headers...
http://hl7.org/implement/standards/fhir/search.html#errors
- Prefer: handling=strict: Client requests that the server return an error for any unknown or unsupported parameter
- Prefer: handling=lenient: Client requests that the server ignore any unknown or unsupported parameter
http://hl7.org/implement/standards/fhir/http.html#prefer
- Prefer: return=minimal
- Prefer: return=representation
- Prefer: return=OperationOutcome
http://hl7.org/implement/standards/fhir/async.html#3.1.6.2.0.1
- Prefer: respond-Async
Specifically how you might request for search strict and Async.
Paul Church (Mar 19 2019 at 00:50):
Per https://tools.ietf.org/html/rfc7240 section 2, either comma-separated or two separate Prefer headers are both accepted.
Brian Postlethwaite (Mar 19 2019 at 00:53):
Thanks.
Last updated: Apr 12 2022 at 19:14 UTC
