FHIR Chat · US Core date search with comparator · argonaut

Stream: argonaut

Topic: US Core date search with comparator

Yunwei Wang (Aug 06 2021 at 21:07):

Inferno received a question from a server implementer about the comparators in Procedure.date search.
The FHIR seach page says that

a prefix to the parameter value may be used to control the nature of the matching ... If no prefix is present, the prefix eq is assumed.

US Core 3.1.1 Procedure.date search parameter has (

comparator : eq ( Conformance Expectation = MAY)

So the server reject request like /Procedure?date=2021&patient=1 based on the reason that without comparator mean eq and eq comparator is not required
Is that the correct interpretation?
@Eric Haas

Eric Haas (Aug 06 2021 at 23:28):

??? I don't understand. MAY means the server may support the comparator eq:

1. if it does you can elect to use it or not it doesn't change the search outcome.
2. if it does not you can elect to use it and the server won't know what to do with it.

Eric Haas (Aug 06 2021 at 23:33):

Also US Core is effectively silent on the MAYs, we are defining the SP for the sake of the CapabilityStatements.

Yunwei Wang (Aug 07 2021 at 01:10):

The question is on the date search with comparator. From this

GET [base]/Procedure?patient={Type/}[id]&date={gt|lt|ge|le}[date]{&date={gt|lt|ge|le}[date]&...}

A server shall support searching by date without comparator GET [base]/Procedure?patient=[id]&date=[date]

If a server interpret "MAY support date search witheq comparator" as "MAY support date search without comparator", then a server can reject any date search without comparator.

Eric Haas (Aug 07 2021 at 04:28):

I don't understand the logic here, we are laying down the minimum requirements, servers can layer additional stuff on it but that does not obviate these requirements.

Yunwei Wang (Aug 07 2021 at 15:41):

Let me ask in another way. Per US Core v3.1.10 Procedure profile, is server required to implement this search GET [base]/Procedure?patient=[id]&date=[date]?

Josh Mandel (Aug 07 2021 at 16:07):

Per

SHALL support searching using the combination of the patient and date search parameters:

?! This seems clear. Yunwei, can you spell out your logic/question step by step?

Josh Mandel (Aug 07 2021 at 16:09):

Is the question whether a server needs to support state searches with no comparator? The answer seems to be a pretty obvious yes to me, since that's the default kind of data search and date searches required.

Josh Mandel (Aug 07 2021 at 16:12):

(I will say the phrase "SHALL support searching using the combination of the patient and date search parameters
....including optional support for composite AND search on date" is problematic for me:. First, fhire has something different called composite search, and while this uses the word composite I don't think it's talking about fhir composite searches it is just talking about repeating a search parameter. Second, I don't know what the word optional means here, within the context of a SHALL. Is this like saying MAY?)

Eric Haas (Aug 08 2021 at 00:07):

Josh Mandel said:

(I will say the phrase "SHALL support searching using the combination of the patient and date search parameters
....including optional support for composite AND search on date" is problematic for me:. First, fhire has something different called composite search, and while this uses the word composite I don't think it's talking about fhir composite searches it is just talking about repeating a search parameter. Second, I don't know what the word optional means here, within the context of a SHALL. Is this like saying MAY?)

This is what is says in the Quick start:

image.png

source: https://www.hl7.org/fhir/us/core/StructureDefinition-us-core-procedure.html#mandatory-search-parameters

source: https://www.hl7.org/fhir/us/core/StructureDefinition-us-core-procedure.html#mandatory-search-parameters

Josh Mandel (Aug 08 2021 at 00:48):

(Yes the image you included is a snapshot of the same text that I quoted. I'm saying the text is problematic for me because it puts an "optional" requirement directly within the body of a SHALL requirement in a section with the word "mandatory" in the name. Optional for whom, and how is it different from a "MAY")

Yunwei Wang (Aug 08 2021 at 02:11):

Josh Mandel said:

Is the question whether a server needs to support state searches with no comparator? The answer seems to be a pretty obvious yes to me, since that's the default kind of data search and date searches required.

That is my interpretation too but I was different one in a practice. Thanks to confirm my thought.

Yunwei Wang (Aug 08 2021 at 02:15):

Josh Mandel said:

(Yes the image you included is a snapshot of the same text that I quoted. I'm saying the text is problematic for me because it puts an "optional" requirement directly within the body of a SHALL requirement in a section with the word "mandatory" in the name. Optional for whom, and how is it different from a "MAY")

My reading is server SHALL do A and MAY do B as addition. I don't see a problem for that.

Josh Mandel (Aug 08 2021 at 04:36):

(I just find "SHALL support... optional composite" to be confusing. I don't think it's a problematic requirement if it just means "MAY support repeated date parameters".)

Stephen MacVicar (Aug 09 2021 at 12:08):

A server shall support searching by date without comparator

@Yunwei Wang Where are you seeing this?

Yunwei Wang (Aug 09 2021 at 13:04):

@Stephen MacVicar

From GET [base]/Procedure?patient={Type/}[id]&date={gt|lt|ge|le}[date]{&date={gt|lt|ge|le}[date]&...}.
Variables surrounded by “{}” are optional for the client to supply
removing all {...}, you get the base form: GET [base]/Procedure?patient=[id]&date=[date]

Stephen MacVicar (Aug 09 2021 at 13:11):

Then it seems like an error that support for eq is a MAY rather than a SHALL.

Eric Haas (Aug 09 2021 at 16:09):

@Stephen MacVicar it is not an error, that is intentional, the pattern is to define the SHALL and SHOULDS and everything else defaults to MAY. Why do you think it was an error?

Eric Haas (Aug 09 2021 at 16:10):

@Josh Mandel I left mid thought above .... I think you may have a valid point but let me dust off the reasoning (or lack of it ) when creating the text templates for this section...

Eric Haas (Aug 09 2021 at 16:20):

First the use of the term 'composite' is overloaded. We try to be consistent with the language used in the base FHIR spec and this term the the concept of 'joining single values with a $' comes from here:

http://build.fhir.org/search.html#combining

and is defined using the SearchParamenter.multipleAnd parameter. This is the intent of its use in the guidance quoted above.

As you state is can also refer to a type of search parameter that take sequences of single values that match other defined parameters as an argument.

Stephen MacVicar (Aug 09 2021 at 16:25):

If servers have to support date search without a prefix, but supporting eq is a MAY, that means that servers have to support the eq functionality when no prefix is sent, but don't have to support it when the eq prefix actually is sent.

Eric Haas (Aug 09 2021 at 16:26):

@Stephen MacVicar that is correct

Eric Haas (Aug 09 2021 at 16:35):

multipleAnd conformance expectation on date is a SHOULD and hence why shows up as an optional in the text. SHOULDs are not enforceable and the same as a MAY from a testing perspective. ( Just like example and preferred bindings are functionally the same) But we felt it necessary to call it out. @Josh do you have any suggestion how to rephrase it in the template to make it Clearer noting that there are SHALL and SHOULD sections and we don't want the same Search Parameter in both sections.

Eric Haas (Aug 09 2021 at 16:41):

Is this an improvement?

Mandatory Search Parameters:

The following search parameters and search parameter combinations SHALL be supported:

SHALL support searching for all procedures for a patient using the patient search parameter:

GET [base]/Procedure?patient={Type/}[id]

Example:

GET [base]/Procedure?patient=1291938
Implementation Notes: Fetches a bundle of all Procedure resources for the specified patient (how to search by reference)

SHALL support searching using the combination of the patient and date search parameters:

• including support for these date comparators: gt,lt,ge,le
• including optional support for composite AND search on date (e.g.date=[date]&date=[date]]&...)
  GET [base]/Procedure?patient={Type/}[id]&date={gt|lt|ge|le}[date]{&date={gt|lt|ge|le}[date]&...}

CHANGE TO:

• Note: support for composite AND search on date is recommended...

...

Stephen MacVicar (Aug 09 2021 at 16:44):

If support for date searches without a prefix is a requirement (which I think is a good idea), it should be stated more explicitly in the spec. I think it's too easy to look at the search parameter details, see eq as a MAY, and come to the conclusion that only date searches with the SHALL comparators need to be supported.

It could be the case that this is a base FHIR requirement, in which case that would be an appropriate place to address this, as the language there is also not very clear.

Eric Haas (Aug 09 2021 at 16:45):

@Stephen MacVicar feel free to make a change request, I would recommend it be conformance expectation of SHOULD search requirements including modifiers need consensus from the community.

Eric Haas (Aug 09 2021 at 16:50):

@Stephen MacVicar I agree this is assumed from the base FHIR requirements and that additional guidance would be helpful.

Stephen MacVicar (Aug 09 2021 at 16:57):

I'll start a thread in implementers later today

Eric Haas (Aug 09 2021 at 16:59):

The discussion will get much more attention as a US Core tracker, I recommend making the tracker first and we can discuss as part of next US Core release.

Stephen MacVicar (Aug 09 2021 at 17:27):

Tracker item: https://jira.hl7.org/browse/FHIR-33156

Josh Mandel (Aug 09 2021 at 17:47):

support for composite AND search on date is recommended

This term "composite" is still throwing me off. Do we mean multipleAnd (i.e., repeating the date param to specify an intersection)?

Josh Mandel (Aug 09 2021 at 17:48):

And why "recommended" for this, but "SHOULD" for the requirements above? What is the distinction between "recommended" vs "SHOULD"?

Eric Haas (Aug 09 2021 at 18:03):

Josh Mandel said:

support for composite AND search on date is recommended

This term "composite" is still throwing me off. Do we mean multipleAnd (i.e., repeating the date param to specify an intersection)?

as I explained above, this term is overloaded in the FHIR specification, we use the first way is it defined as referenced above. What do you suggest?

Eric Haas (Aug 09 2021 at 18:08):

Josh Mandel said:

And why "recommended" for this, but "SHOULD" for the requirements above? What is the distinction between "recommended" vs "SHOULD"?

We try to avoid Nesting conformance verbs (a SHOULD under a SHALL) to be clear and avoid confusion. We think that is even more confusing than providing additional guidance in more informal language.

Josh Mandel (Aug 09 2021 at 18:53):

["composite"] is overloaded in the FHIR specification, we use the first way is it defined as referenced above. What do you suggest?

@Eric Haas I don't see any overloading -- http://build.fhir.org/search.html#combining defines three distinct concepts:

• "AND search parameter" -- e.g., Patient?language=FR&language=NL for French/Dutch bilingual patients
• "OR search parameter" -- e.g., Patient?language=FR,NL for patients who speak at least one of French or Dutch
• "Composite search parameter" -- e.g., Group?characteristic-value=gender$mixed for Groups with a single characteristic that means "gender: mixed" (rather than, say, one characteristic meaning "gender: male" and another meaning "preferred Chex cereal format: mixed", which you might inadvertently match with a non-composite query likeGroup?characteristic=gender&value=mixed).

(Edit: I do see why it's confusing that all three are introduced in a section called "Composite Search Parameters")

Eric Haas (Aug 09 2021 at 19:01):

"Composite search parameter" is defined as:

Composite search parameters support joining single values with a $. For example, the result of the search operation is the intersection of the resources that match the criteria specified by each individual search parameter.

Eric Haas (Aug 09 2021 at 19:06):

My point is however, we consciously tried to use the same language as in the base specification, any confusion here reflects lack of clarity in the FHIR standard. We can change it to make it clearer in both places

Josh Mandel (Aug 09 2021 at 19:18):

Composite is defined as joining with "$". Which isn't something we're doing with dates. Agreed that the core spec clarity can be improved, but I don't see any definitional issue.

Eric Haas (Aug 09 2021 at 20:08):

Composite is defined as joining with "$". Which isn't something we're doing with dates.

To be clear we are recommending doing this with dates Get /Condition?_id=123&date=ge2020&date=le2021

Josh Mandel (Aug 09 2021 at 21:20):

I'm just saying we're not talking about using any FHIR search params whose SearchParameter.type is composite.

Re: what we do need... what are the use cases we care about? I think asking for onset-date=2020 or onset-date=eq2020 today would find any Conditions with an onset in 2020, so we wouldn't need to repeat params (like onset-date=ge2020&onset-date=le2021) just for this.

• Where you could get "something new" would be to express things like "Conditions diagnosed in 2018 or 2020" (this is not a compelling use case though).

• There's a bug in the example using "date" instead of "onset-date"; filed FHIR-33157

Eric Haas (Aug 09 2021 at 22:20):

re :

There's a bug in the example using "date" instead of "onset-date"; filed FHIR-33157

I just threw that example together here to illustrate

is not a bug in the spec:

image.png

Eric Haas (Aug 09 2021 at 22:23):

what we do need... what are the use cases we care about?

... that is what we spent the last several years doing.... I don't think that is the issue here

Eric Haas (Aug 09 2021 at 22:28):

I still assert that he term "composite" is not limited to a type of search parameter as defined by FHIR, but that is also tangential. Will consider a tracker to make the intent clearer and avoid the ambiguous FHIR jargon.

Josh Mandel (Aug 09 2021 at 22:49):

is not a bug in the spec

@Eric Haas screenshot includes the bug (at the very bottom, after a couple of valid examples):

image.png

See where the current IG says date instead of onset-date? Or am I just confused?

Josh Mandel (Aug 09 2021 at 23:00):

what we do need... what are the use cases we care about?
... that is what we spent the last several years doing.... I don't think that is the issue here

Sure, I'm not asking to relitigate any decisions. I'm just trying to understand what is the use case for repeated "date" parameters? (I think it's fine to call out as an optional thing servers can choose to support; I'm just genuinely curious about why we are calling it out.)

Eric Haas: I still assert that he term "composite" is not limited to a type of search parameter as defined by FHIR, but that is also tangential. Will consider a tracker to make the intent clearer and avoid the ambiguous FHIR jargon.

Sounds good, submitted FHIR-33159 for this (and FHIR-33159 to clarify the underlying FHIR spec)

Eric Haas (Aug 10 2021 at 01:36):

1. yeah its a bug.... I did not read it to the end.
2. the intent of supporting ANDs is so the date query get a specific period of time in the past.

Eric Haas (Aug 10 2021 at 01:37):

It is indeed more of a "nice to have" feature and thus is a SHOULD vs SHALL.

Josh Mandel (Aug 10 2021 at 02:03):

Cool. FWIW i don't think ANDs really do help date queries get a specific time in the past. Anything you might say by "AND"ing two dates (assuming some nonzero overlap) is going to be identical to using just one input (whichever is narrowest).

Last updated: Apr 12 2022 at 19:14 UTC