Stream: implementers
Topic: Unsupported search parameters
Heath Frankel (May 11 2016 at 03:13):
The specification is clear that unsupported search parameter modifiers should be rejected, but I couldn't find any guidance about unsupported search parameters. Should unsupported search parameters result in the request being rejected or is the parameter ignored and the result returned as though the parameter was not provided?
Lloyd McKenzie (May 11 2016 at 03:21):
Unsupported search parameters are ignored. You echo back what search parameters you actually use in the location header
Josh Mandel (May 11 2016 at 03:21):
http://hl7-fhir.github.io/compatibility.html#1.10.3
Josh Mandel (May 11 2016 at 03:22):
In theory, we specify that:
Unrecognized search criteria SHALL always be ignored
In practice some popular servers don't find this approach palatable.
Josh Mandel (May 11 2016 at 03:23):
Also, the only way I was able to find that quote was by knowing it existed and googling carefully. It's totally undiscoverable information.
Josh Mandel (May 11 2016 at 03:24):
On search.html we say:
server SHALL reject any search request that contains is suffixed by a modifier that the server does not support
but we say nothing about "unrecognized search criteria".
Heath Frankel (May 11 2016 at 03:52):
Glad to know I wasn't the only one that couldn't find it. I had a feeling I had read it before but couldn't find it when I reread looking for it again.
Grahame Grieve (May 11 2016 at 10:43):
what we specifically say is that don't reject a search because of unknown url parameters.
Grahame Grieve (May 11 2016 at 10:44):
we do not make any rules about recognised but unsupported search parameters. A server can ignore them, return a bigger result set than expected, and let the client now in the _self link. or it can refuse to perform the search.
Grahame Grieve (May 11 2016 at 10:44):
I do not think that forcing all servers to do one or the other is a good thing
Grahame Grieve (May 11 2016 at 10:44):
(deleted)
Michael Donnelly (May 11 2016 at 11:51):
I agree strongly. :)
Grahame Grieve (May 11 2016 at 12:15):
http://gforge.hl7.org/gf/project/fhir/tracker/?action=TrackerItemEdit&tracker_item_id=9985
Sean Moore (May 11 2016 at 18:03):
After talking with @Michael Donnelly for a bit in the real world, I would say that I'm more in favor of at minimum specifying the servers SHOULD ignore these parameters rather than return an error. From a client perspective, allowing servers to error on unsupported search parameters means that clients need to bake in behavior to avoid errors emitted from servers due to differing search parameter support. by saying servers should ignore, the likely case is that clients will get additional data back that they did not expect, and good clients will filter themselves anyway.
Sean Moore (May 11 2016 at 18:04):
Ignoring the parameters seems far more likely to allow clients to work across server implementations
James Agnew (May 11 2016 at 18:40):
I continue to be surprised that so many people would want servers to more-or-less silently ignore search parameters they don't understand. I mean yeah, you can figure it out by examining the self link, but the behaviour "you asked for X but I'm going to give you Y instead because I don't understand X" just seems to me like a recipe for bugs (at best), or just plain unsafe at worst.
Ultimately I'm glad we're in agreement that this should not be prescriptive on the part of FHIR. I guess this should really be configurable in HAPI.
Chris Moesel (May 11 2016 at 19:20):
I agree with @James Agnew -- it's dangerous for a server to return an answer that is not for the question it was asked. I think that in something as life-and-death as healthcare, we should not design on the assumption that clients will employ well-written, defensive code. <horrible-analogy> Sam has a gluten allergy. Sam goes to the store and buys a box of cookies from the "gluten-free" aisle. He eats them and gets sick. When he complains to the store, they tell him that even though the cookies were in the gluten-free aisle, it's his fault because he should have read the ingredients label. </horrible-analogy>. Aren't supported search params part of the conformance statement? If so, then the "good clients" should already know not to issue the search anyway -- and the "not-so-good clients" will get the error they deserve (and that won't harm anyone).
John Moehrke (May 11 2016 at 19:25):
I think this is recognition that in REST a query is an ask of the server to do the best job it can to find the results, and not an expectation that the query is a perfect query that produced absolute results.
James Agnew (May 11 2016 at 19:27):
Where does that principle come from?
Kevin Shekleton (May 11 2016 at 19:34):
@James Agnew - This is the behavior of every website that I can think of when I pass it a query parameter that it doesn't understand. It just ignores it because it isn't looking for that particular param. Eg: http://www.google.com/search?btnG=1&pws=0&q=fhir&unknownParameter=foo
James Agnew (May 11 2016 at 19:37):
I feel like using how websites work isn't the best way of informing our choices..... I mean, google's homepage isn't a REST API, it's a website. They want to maximize eyeballs on their page, not maximize safe use of their data.
Kevin Shekleton (May 11 2016 at 19:39):
Well, that is what REST is -- the web.
Kevin Shekleton (May 11 2016 at 19:40):
Part of the reasoning for using REST (and the success of it) is the conventions that are part of REST and what make it nice/easy to work with as a developer
Michele Mottini (May 11 2016 at 19:41):
I think there is a difference between an unknown search parameter (not listed in the specs) - that is quite safe to ignore - and a parameter that is in the spec but the server does not implement
Grahame Grieve (May 11 2016 at 19:41):
I do believe that so far we have found strong support for my contention that we should not force behaviour either way
Kevin Shekleton (May 11 2016 at 19:44):
@Michele Mottini - Good point. A well designed server will advertise the search params it honors in the Conformance and a well designed client will check this
James Agnew (May 11 2016 at 19:45):
Heh, on that there is no doubt.
Chris Moesel (May 11 2016 at 19:49):
Yeah, it's the assumption of a "well designed client" that concerns me. In so many spaces, it doesn't matter too much if the server "does its best"-- but I feel that in this space, a well-intentioned best effort can lead to disastrous results.
Grahame Grieve (May 11 2016 at 19:50):
it depends on the search parameter.
Grahame Grieve (May 11 2016 at 19:51):
ignoring subject/patient search parameter is probably going to be a disaster, but ignoring some qualifier will lead to too many matches in a way that is obvious to the user immediately
Chris Moesel (May 11 2016 at 19:52):
/AllergyIntolerance?substance=http://snomed.info/sct| 227493005&status=refuted
Deadly if status is not supported.
Chris Moesel (May 11 2016 at 19:54):
Given that so many fhir resources can have their meaning totally changed (or even inverted) by a single attribute, there are many examples where this can be very bad.
Kevin Shekleton (May 11 2016 at 20:01):
1. This is the predominant convention of the web -- ignore unknown query params
2. You're calling out a poorly designed client which is the root problem -- not the lack of an error on the unhandled status param. If the client is getting this wrong by wrongly assuming the server can handle status, it's probably missing a bunch of other stuff too
James Agnew (May 11 2016 at 20:06):
(disclaimer here: I'm happy with where we've landed, and I'm certainly not trying to feed a war or change anyone's mind.. I just find this topic interesting)
I don't see why web conventions apply here. REST APIs and websites use the same underlying protocol, but that doesn't mean they are the same thing. Do Google's AdWords APIs ignore things they don't understand? (maybe they do, i have no idea)
Kevin Shekleton (May 11 2016 at 20:11):
I don't know if Google's AdWords API ignores them but I'd bet you a beer they do. :-)
Every JSON API I've seen ignores any key that it doesn't know about
Grahame Grieve (May 11 2016 at 20:13):
Strictly, this is not a fair comparison - google are not choosing to implement a spec they didn't write, and interop spec, They are - as we already recommend - ignoring unknown parameters
Grahame Grieve (May 11 2016 at 20:13):
but I stil agree with Kevin in general
Chris Moesel (May 11 2016 at 20:14):
@Kevin Shekleton: I'm OK with ignoring truly unkown parameters, but as @Michele Mottini noted -- there is a difference between unknown (not in the spec) and unsupported (in the spec but not supported). And yep -- I am calling out a poorly designed client because I have a hunch that if we look at all of the clients making queries to fhir, we'd find more than a few that neither check the server conformance nor check the self url to ensure support of the parameters they are using. But it is only a hunch (I haven't researched it).
Kevin Shekleton (May 11 2016 at 20:15):
I agree with you @Chris Moesel, I'm sure there are lots of badly behaving clients out there. Bugs exist anywhere, even in (unfortunately) our space (healthcare)
Chris Moesel (May 11 2016 at 20:16):
I supposed there is the flipside -- if we say servers must return an error for unsupported parameters, then badly behaved servers (who don't return errors) will cause problems for technically spec-compliant clients.
Jenni Syed (May 11 2016 at 20:17):
Also - defining what "truly unknown" is becomes problematic. What if my poorly behaving client is running an STU 3 client against a DSTU 2 server. They can check conformance for that as well, but that's also where the param support is documented :)
Corey Spears (May 11 2016 at 20:20):
Does this mean someone owes someone else a beer?
https://developers.google.com/doubleclick-search/v2/standard-error-responses#BAD_REQUEST
Kevin Shekleton (May 11 2016 at 20:20):
@Chris Moesel - Note that we could take this entire discussion and apply it to FHIR resources and a 404 response. In this case the client doesn't know if it got a 404 because the FHIR resource isn't implemented in the server (since we're assuming they are poorly designed client that isn't checking Conformance) or the server cannot find the resource.
Kevin Shekleton (May 11 2016 at 20:20):
Maybe :-)
Kevin Shekleton (May 11 2016 at 20:22):
It does look like it. This is the first API I've seen with this behavior. Still doesn't change my opinion on this, but I'll honor that beer bet with @James Agnew nevertheless :-)
Chris Moesel (May 11 2016 at 20:24):
Two beers?
https://msdn.microsoft.com/en-us/library/azure/dd179357.aspx
UnsupportedQueryParameter: One of the query parameters specified in the request URI is not supported.
Kevin Shekleton (May 11 2016 at 20:25):
Github's API ignores unknown params so -1
Grahame Grieve (May 11 2016 at 20:25):
just because they have an error code doesn't mean they actually raise it on a search, for instance
Chris Moesel (May 11 2016 at 20:30):
@Kevin Shekleton: I think the 404 response is a little different, in that 404 is usually returned as the result of the client asking for something specific (by ID) that it believes exists. So if it gets a 404, then it probably suspects something is off since it really expected something. That's a little different than a server returning 10 results versus 8. As far as I recall, FHIR search never returns a 404, but I could be wrong...
@Grahame Grieve and @Kevin Shekleton : Yeah. I was actually about to admit that when I quickly googled I found several well-regarded APIs that indicated they ignored unknown parameters. But... when they own the API, then unknown parameters tend to only be unspecified parameters. It would be weird for Google to create a REST API and then not support the parameters it specified itself.
BTW-- I've made the arguments I wanted to make. I will leave you in peace. ;-)
Jenni Syed (May 11 2016 at 20:39):
From Google calendar (I can try the add ones in a sec): GET /calendar/v3/users/me/calendarList?maxResults=10&holyfoocow=100 HTTP/1.1
Response: HTTP/1.1 200 OK
Content-length: 6628 ...
Grahame Grieve (May 11 2016 at 20:40):
"holyfoocow"? How do you know this is not supported?
Jenni Syed (May 11 2016 at 20:40):
true... ;)
Grahame Grieve (May 11 2016 at 20:40):
that was nasty, actually, I can't really laugh out loud in a committee meeting
Michel Rutten (May 11 2016 at 20:41):
LMAO
Jenni Syed (May 11 2016 at 20:43):
I can't try out double-click specifically because I don't have an account :) If someone does, I can tell you how to try it. You can choose evilfoocow as your extra param if you so want
Chris Moesel (May 11 2016 at 20:46):
Ha. @Jenni Syed, I don't think that will be necessary! I have the feeling that the results of that experiment won't really affect the outcome of this discussion either way. On the other hand, if that's how we have fun around here (issuing evilfoocow queries), then don't let me get in anyone's way! ;-)
Jenni Syed (May 11 2016 at 20:49):
it will be a holyfoocow and evilfoocow duel
Grahame Grieve (May 11 2016 at 20:49):
I think I might have to add something to my server.
Chris Moesel (May 11 2016 at 20:50):
Don't leave out the agnosticfoocow.
Grahame Grieve (May 11 2016 at 20:50):
if url.hasParameter('evilfoocow')
throw new Exception("Jenni, please do not hack with the server, it's very fragile today");
Christiaan Knaap (May 11 2016 at 20:52):
If we can't decide between returning an error or ignoring parameters, there is an option to leave it to the client.
I see three types of clients (for this discussion):
1. Client written for a local setting, that will always query the same server. Returning an error is appropriate, since the client should be aware of the server's capabilities.
2. Client written to target potentially any server (like an app), subdivided in:
2a: Potentially leading to severe injury when search results are not checked properly. Returning an error is helpful here.
2b: Not leading to severe injury, but the maker just wants her/his app to run against any server without bothering the user with errors. A server could help the maker by allowing a 'loose' mode.
To cater for the cases 1 and 2a, a 'strict' mode should be the default. So a server should at a minimum return an error on unsupported parameters.
The loose mode is optional for servers to support, and the client has to ask for it specifically.
But it would be yet another addition to the already rich search api.
Christiaan Knaap (May 11 2016 at 20:54):
PS: I don't mind to rename the modes to holyfoocow and evilfoocow, but I'm still figuring out which should mean 'strict'.
Josh Mandel (May 11 2016 at 20:59):
Yeah I just spent too long trying to sign up for adwords to see what they actually *do*. But I've given up :p
Kevin Shekleton (May 11 2016 at 21:00):
Me too. :-) It's an extremely complicated process involving manager production accounts, test accounts, manager API keys, test API keys. Ugh
Josh Mandel (May 11 2016 at 21:01):
So U have learned. I got all the way to a developer token and a local download of the python API. I'm still not sure it's not a SOAP API (one of their docs mentions the 'suds' python package).
Kevin Shekleton (May 11 2016 at 21:01):
Ha, I went the route of cloning their Ruby & Rails API and was setting up the example server
James Agnew (May 11 2016 at 21:58):
FINE FINE FINE you all win.
Holyfoocow has been implemented: http://fhirtest.uhn.ca/baseDstu3/Patient?holyfoocow=true
I'll be proposing this as a stream at the baltimore connectathon, testing partners welcome.
Grahame Grieve (May 11 2016 at 23:31):
James and Jenni win the FHIR prize
Lloyd McKenzie (May 12 2016 at 01:37):
Could we have a behavior where, by default, servers error if they receive an unexpected search parameter, but you can send an extra search parameter (e.g. softFail=true or something) that causes the server to ignore extra search parameters? That way clients explicitly tell the server to ignore extra parameters and therefore explicitly take responsibility for checking what was actually processed?
Lloyd McKenzie (May 12 2016 at 01:37):
I think that approach keeps the client in charge of behavior, which is desirable, and also helps ensure safety.
Stephen Royce (May 12 2016 at 03:07):
What's the default behaviour for servers that don't support softFail?
Lloyd McKenzie (May 12 2016 at 03:27):
They'd have to raise an error when receiving unexpected search parameters
Stephen Royce (May 12 2016 at 03:29):
And you don't see any inherent contradiction in this behaviour?
Lloyd McKenzie (May 12 2016 at 03:30):
The default is the safe behavior
Stephen Royce (May 12 2016 at 03:46):
So there's no problem then with a query that would have otherwise been successful, but now fails because of the addition of "shotFail-true"?
Michael Donnelly (May 12 2016 at 14:11):
I prefer less optionality rather than more. If a client wants to be sure that what was returned is what it asked for (effectively, hardFail=true), the client can compare its search URL to the one returned by the server.
Michael Donnelly (May 12 2016 at 14:12):
I'm concerned about having two different options for servers, sort of for the same thing that concerns people about soft failing.
Michael Donnelly (May 12 2016 at 14:12):
If some servers - particularly reference servers - return errors, a client may assume that will always happen.
Josh Mandel (May 12 2016 at 14:12):
Yeah, this whole thread of discussion seems over the top to me.
Michael Donnelly (May 12 2016 at 14:13):
If all servers ignore unknown or unsupported parameters, clients are less likely to make this error.
Lloyd McKenzie (May 12 2016 at 14:31):
Most test servers support most parameters. As well, the failure to pay attention to inbound parameters isn't necessarily immediately noticable when you're testing. So I don't think we can count on all client implementers picking up on this "feature" before they hit production. As well, clients created to talk to a specific server may work just fine until the server's support changes (or even changes what name they use for a given parameter - which can happen too). So I think "failing safe" is important here.
Jenni Syed (May 12 2016 at 15:00):
@James Agnew Bonus points for using the I'm a Teapot http error status
Grahame Grieve (May 12 2016 at 15:29):
So I think we need to be clearer about this at the top of the search page, but otherwise the discussion bears out my original statement: we won't get to consensus on this
Lloyd McKenzie (May 12 2016 at 17:26):
Well, non-consensus on this is even less safe - because clients will expect one behavior and may get another and have no control over what happens. Why would we *not* want to leave the choice of behavior in the client's hands? The only reasons for driving server behavior I've heard so far is "what's typical" and "safety" - I don't see why those two concerns couldn't be addressed by giving the client the control of what happens. Are there any technical reasons why a server *couldn't* error on or *couldn't* ignore unsupported extensions if the client told them their desired behavior?
Corey Spears (May 12 2016 at 17:58):
Personally I side with Lloyd. The inherent contradiction of a server providing an error on an invalid parameter (softFail) when it does not support erring on invalid parameters is not lost on me. Pragmatically, in that case, the requirement is that all servers would have to support an indication as to whether softFail is supported.
We are talking about health information here. Much more critical than many APIs dealing with Ads and the like. We certainly can debate the details, but it seems some sort of feedback regarding the validity of requests is warranted.
Jenni Syed (May 12 2016 at 18:05):
Keep in mind that if a server does this, there are still going to be "holes." For example, if we state that a server errors when it sees "known unknown" parameters (parameters it chooses not to implement), what happens when an app is expecting the server to be running a different version of the spec?
Jenni Syed (May 12 2016 at 18:07):
What about custom parameters that the server doesn't know about (AFAIK, there is no "parameter extension" syntax)? EG: when an IG defines a new param?
Jenni Syed (May 12 2016 at 18:09):
I know we say "this is healthcare" but if our stance is to ignore the underlying norms of the transport/architecture we're on top of because we're special, we may get unintended results. Google apis deal with many different aspects of data that have financial and personal privacy impact, so it's not like data there isn't also sensitive
Lloyd McKenzie (May 12 2016 at 18:46):
This doesn't prevent following underlying norms, it just forces an explicit acknowledgment in the instance that the client knows that's what's going on and has taken responsibility for the ramifications. To resolve the contradiction, one possibility is to say that all servers have to support the "softFail" parameter - then you don't have to worry about it not being supported. If a client is wanting to pass extra stuff that the server is going to ignore, they have a choice - trim those before going to the server or make sure their client detects what clients are paid attention to.
James Agnew (May 12 2016 at 18:53):
Despite my belief that ignoring unknown parameters is the wrong default, this probably isn't a spot where we could make a hard and fast rule.
There are going to be times when the client doesn't even know it is adding parameters. Lots of reverse proxies add things along the way, caching or session-pinning mechanisms may too..
Grahame Grieve (May 12 2016 at 18:53):
there's a difference between unknown parameters and unsupported search parameters
Jenni Syed (May 12 2016 at 18:57):
There is, but there will be "unknown FHIR parameters" IE: A paramenter added in an IG that the server isn't aware of, a client running a different version of the FHRI spec that the server, etc...
Lloyd McKenzie (May 12 2016 at 18:59):
Right. There's no way of knowing what a server knows about and doesn't support.
Joel Schneider (May 16 2016 at 19:09):
+1 for Lloyd's softFail idea, making "failing safe" (or failing fast) the default. The alternative seems dangerously similar to exception eating.
Grahame Grieve (May 16 2016 at 19:11):
there is a way of knowing what a server supports. Maybe we should say that a server cannot ignore a parameter it claims to support in it's conformance statement.
Ewout Kramer (May 19 2016 at 09:57):
Or maybe we should look into how we could return an OperationOutcome in a bundle. I know Lloyd has been asking for it, and it keeps popping up. A server could at least then return a warning status with a specific code about the search parameter.
Grahame Grieve (May 19 2016 at 21:49):
well, we could add a new search mode 'commentary'
Peter Scholz (May 20 2016 at 10:34):
IMHO it is potentially dangerous to ignore unknown parameters.
so I would still leave it up to the server to decide how to handle this situation.
But on the other hand, giving the client the opportunity to suggest a behaviour would be a good idea.
But if we do so, I think there is a better place than an additional parameter.
I would prefer to designate this using a
Prefer: handling=strict resp. handling=lenient in the header as described in
https://tools.ietf.org/html/rfc7240#section-4.4
Grahame Grieve (May 20 2016 at 11:27):
interesting idea
Last updated: Apr 12 2022 at 19:14 UTC
