Stream: implementers
Topic: Using hyphen (-) in search parameters
Omar El-Begawy (Aug 08 2017 at 07:02):
Hi, the latest version of FHIR uses hyphens in the search parameters of some profiles, e.g. Patient has address-city and general-practitioner. However, using hyphens in query parameters does not follow the RFC6570 specification. Libraries that follow this specification strictly will give an error message when they encounter a query with a hyphen, e.g. this library https://github.com/grncdr/uri-template . Is there a way to update the FHIR specification to follow the URI specification? This affects how data models are constructed and REST queries that implementations can make.
Grahame Grieve (Aug 08 2017 at 08:46):
does rfc 6570 matter?
Peter Scholz (Aug 08 2017 at 09:01):
since when is a hyphen not allowed in an URI ?
I don't see that anywhere
Jose Costa Teixeira (Aug 08 2017 at 10:17):
My understanding is that 6750 simply provides an additional syntax to express some queries. And in that additional syntax hyphens are not part of the syntax. URIs allow hyphens, but RFC6750-formatted queries (they call them "URI templates" do not. If that is correct, I'd say the question is: do we want to provide additional rfc 6750 syntax ? It does not affect current specification IMO. I may be wrong.
Omar El-Begawy (Aug 08 2017 at 12:38):
We are testing our API implementation with REST requests, but the test tool uses a validation library which considers query parameters with hyphens to be malformed. We found that the reason was that they did not conform to RFC6570.
This could be a problem also with other tools that closely follow RFC 6570.
URIs can take many different forms, some even arbitrary ones. RFC 6570 is an attempt to standardize their formats and make them well-formed.
Here is the specification: https://tools.ietf.org/html/rfc6570
A list of implementations: https://code.google.com/archive/p/uri-templates/wikis/Implementations.wiki
Eric Haas (Aug 08 2017 at 14:42):
what is recommmended instead of hyphens?
Omar El-Begawy (Aug 09 2017 at 05:25):
The spec gives these:
varchar = ALPHA / DIGIT / "_" / pct-encoded
where
ALPHA = %x41-5A / %x61-7A ; A-Z / a-z
DIGIT = %x30-39 ; 0-9
pct-encoded = "%" HEXDIG HEXDIG
HEXDIG = DIGIT / "A" / "B" / "C" / "D" / "E" / "F"
; case-insensitive
I'd say it's up to you to select from this set.
I saw that you had camelcasing at some point. What happened to that?
Grahame Grieve (Aug 09 2017 at 22:57):
we had a mix of camel casing and hyphens. We standardised on hyphens.
Grahame Grieve (Aug 09 2017 at 22:58):
I have no idea why this rfc decided that hyphens are not approprate - it seems like an odd decision to me
Omar El-Begawy (Aug 10 2017 at 13:25):
I have no idea either. But my guess is that either the spec writing group did not have a use case where the hyphen was part of a query parameter and therefore did not even think that it could be an issue, or the hyphen is unreserved until given a role as a special character in the future.
Peter Scholz (Aug 10 2017 at 14:06):
i don't see any rationale why variable names should be treated different than all other parts of the URI
as far as I understand variable names are the only parts that have different rules
IMHO there is no real benefit in adding the restrictions of RFC 6750.
Peter Scholz (Aug 10 2017 at 14:13):
@Eric Haas there is no "instead" it would just mean, that submitting the query would need to escape the hyphen like
GET ..../Patient?adress%2dcity=New%20York
Peter Scholz (Aug 10 2017 at 14:31):
I did just sent such an encoded query to Grahame's server and it worked well. So imho there is no need to change the search parameters.
It would be up to the client to encode search parameter names according to RFC 6750 if desired.
Therefore I think there is no need for action regarding the specs
Lloyd McKenzie (Aug 10 2017 at 15:08):
Well, if some servers can handle dashes in search parameters and others can't, that creates a barrier for interoperability. How prevelant are libraries that expect RFC 6750 adherence?
Peter Scholz (Aug 10 2017 at 17:12):
@Lloyd McKenzie just to put that right: RFC 6750 does not forbid dashes in variable names, but enforces escaping on transmission.
That escaping will be undone in the HTTP receiver on the server side before passing the HTTP request to the layer above. Therefore the application will receive "normal" search parameters.
For that reason, RFC 6750 can't be an issue on the server side, but only on the client side.
Lloyd McKenzie (Aug 10 2017 at 20:28):
So perhaps adding a note that some client systems may need to escape the "-" in search parameters is sufficient?
Omar El-Begawy (Aug 11 2017 at 05:53):
We already tried replacing - with %2d. Didn't work.
Peter Scholz (Aug 11 2017 at 07:26):
That should have worked, as that ist what RFC 6750 says,
If it didn't work your library is buggy on which side is the library located server or client ?
Omar El-Begawy (Aug 11 2017 at 13:47):
The spec gives these:
varchar = ALPHA / DIGIT / "_" / pct-encoded
where
ALPHA = %x41-5A / %x61-7A ; A-Z / a-z
DIGIT = %x30-39 ; 0-9
pct-encoded = "%" HEXDIG HEXDIG
HEXDIG = DIGIT / "A" / "B" / "C" / "D" / "E" / "F"
; case-insensitive
The percentage sign is not part of the spec, and therefore %2d is not valid. Your query works because STU3 is not compliant with RFC 6570. But if client applications want to adhere to the spec or they use libraries that do so, there will be queries that they cannot make to a FHIR server. This affects url validation, routing, etc.
Templates and standards are useful because they are general and reusable in different contexts. URIs can very well be arbitrary, but nonstandard URIs are a hindrance to interoperability, because it requires more time for developers to learn their patterns. Nonstandards compliance also makes systems prone to errors.
It would be quite questionable for us to ask application developers and contractors to replace - with %2d, as this cannot be found in the STU3 documentation. With multiple vendors building mobile applications for health care and contractors building services in a microservice environment, it is important to adhere to standards. We cannot guarantee that all parts of our system are interoperable if some follow a standard that others do not, nor can we guarantee that contracted parts of our system will pass the necessary audits.
Omar El-Begawy (Aug 11 2017 at 13:49):
Please reference RFC 6570, which is about URI templates. I am not concerned with RFC 6750, which is about OAuth (earlier there was one question in this stream about OAuth, probably because the person made a query with the OAuth spec reference)
Grahame Grieve (Aug 11 2017 at 20:34):
well, I think you should go to the authors of the library and suggest that they support '-' anyway. Even if we changed policy on this and made breaking changes to all of our resources - something I find unlikely at best - the change would still not be effective till R4 - which won't be published until end of next year and only adopted gradually after that. And none of this discussion has demonstrated that RFC 6570 has any real reason to ban '-'. I smells like an oversight to me.
Last updated: Apr 12 2022 at 19:14 UTC
