Stream: argonaut
Topic: token search?
Drew Torres (Jun 30 2020 at 13:08):
Did we really intend to have to support all token searches?
[parameter]=[code]: the value of [code] matches a Coding.code or Identifier.value irrespective of the value of the system property
[parameter]=[system]|[code]: the value of [code] matches a Coding.code or Identifier.value, and the value of [system] matches the system property of the Identifier or Coding
[parameter]=|[code]: the value of [code] matches a Coding.code or Identifier.value, and the Coding/Identifier has no system property
[parameter]=[system]|: any element where the value of [system] matches the system property of the Identifier or Coding
Drew Torres (Jun 30 2020 at 13:08):
The one I have an issue with is parameter=system.
Drew Torres (Jun 30 2020 at 13:09):
We would basically have to support a search that says give me all practitioners with an NPI value... I don't think we intended that.
Josh Mandel (Jun 30 2020 at 13:16):
Is this referring to the US Core IG?
Drew Torres (Jun 30 2020 at 13:16):
Yes. It links to that for additional information.
Drew Torres (Jun 30 2020 at 13:17):
US-Core is mostly silent on it.... It shows an example of using system|code only, but then links to the token documentation for additional details.
Drew Torres (Jun 30 2020 at 13:17):
Inferno is testing all combinations.
Drew Torres (Jun 30 2020 at 13:18):
Which surprised me because we are only planning on supporting system|code.
Josh Mandel (Jun 30 2020 at 13:22):
Given the prevalence of nonstandard Coding.system values (OIDs, local variation etc) I'd say that searching by code alone can be pretty pragmatic. Searching by system alone... if there's an existing "get all" it's less critical, but this is part of the fhir spec for a reason, and if you're already creating the indexes to make the other forms work, is it an additional lift?
Drew Torres (Jun 30 2020 at 13:26):
We aren't creating indexes to make the others work. Our architecture leverages mappings for codes to internal values.... So a code comes in, we look up all internal codes that map to that, and then begin a query for all codes. To say give me a code with no system is going to slow down performance in a pretty significant way... I am not a huge fan of that query either. The system one just doesn't make sense to me. Why not just use patient and get everything back for that patient in the single patient query? By adding system we will actually make the search slower and cost additional resources to complete the call.
Keith Boone (Jun 30 2020 at 13:28):
Having built this search against a custom database, YES, it is additional lift. Not every implementation has a FHIR core storage. NPI may be field somewhere, and identifier queries may be built on a view that includes that field and others. BUT, I agree that it's completely worthwhile to support (which is why I built it). There probably aren't that many normal user use cases for using this query. However, I can think off a half dozen use cases where IT staff want it, and have used it, to identify and clean up incomplete records in a system.
Josh Mandel (Jun 30 2020 at 13:29):
Re: codes alone, does this mean that any local value without a mapping pointing to it is invisible under search? Even if the "code" matches, @Drew Torres ? That seems worth addressing.
Drew Torres (Jun 30 2020 at 13:30):
Correct. If there is an underlying internal code that cannot be mapped, the only way to get to it would be /Observation?patient=12345 and page through the results.
Drew Torres (Jun 30 2020 at 13:31):
Which we support.
Drew Torres (Jun 30 2020 at 13:31):
There are just things that health system document as observations that just won't ever have standard terms for.
Josh Mandel (Jun 30 2020 at 13:32):
Oh sure, that is well and good. I just want to make sure that when something is documented with a standard code (but a nonstandard system), it's found by a search when the search specifies exactly the code that was documented.
Josh Mandel (Jun 30 2020 at 13:33):
Like, supporting the &code=|123 search helps a lot in that case.
Drew Torres (Jun 30 2020 at 13:36):
I suppose, but is the consumer just going to blindly query with a code and not know the system? That is what I am struggling with in your example. We do allow searches with our internal codes and code systems. But allowing a consumer to just pass in code without a system will incur that additional lookup cost... Not huge and not as bad as system only. That is why I am not pushing back on that one too hard.
Drew Torres (Jun 30 2020 at 13:39):
Isn't there also an argument that it is dangerous to do that as well? If the consumer of the API just passes in code=123.... The internal codes and the standard codes may conflict or mean completely different thing. In that case the consumer should understand the variations... If that is the case why not force the consumer to search with system in addition to code?
Josh Mandel (Jun 30 2020 at 13:41):
If the consistency of "system" in the source data is low enough, clients will skip it; I'm not saying it's a great practice, but it's pragmatic, and it allows a client to over-fetch by only a little bit rather than having to get everything. The client probably still needs to look at systems it receives, but doesn't need to know ahead of time exactly which systems a given server inexplicably uses OIDs for in order to formulate a special search
Josh Mandel (Jun 30 2020 at 13:41):
(when I say inexplicable here, I don't mean there is no explanation. I just mean that it can be baffling for client developers or try to understand the variations in data conventions across sites)
Drew Torres (Jun 30 2020 at 13:43):
That makes more sense.... We really don't have that problem because we centralized our content for mappings. There is no variances of systems/OIDs for content. We always return URLs for systems. We only return OIDs for identifiers on patients for non-standard identifiers.
Drew Torres (Jun 30 2020 at 13:43):
Which is why I struggled with your example, but does make sense for systems that don't have that luxury.
Drew Torres (Jun 30 2020 at 13:47):
We also document all of our systems supported in our documentation so dev should be aware ahead of time: http://fhir.cerner.com/millennium/r4/diagnostic/observation/#terminology-bindings is an example (include local system examples).
John Moehrke (Jun 30 2020 at 15:19):
I have only heard of administrative reasons to support that query. What end-user use-case(s) drive a demand for support, vs a should or may?
Yunwei Wang (Jun 30 2020 at 16:49):
@Josh Mandel I think search by code alone is reasonable since the binding value set has contains on what system should/shall be used.
Robert Scanlon (Jun 30 2020 at 17:13):
@Drew Torres -- are you sending resources with Codings with system but no code? That may cause Inferno to do a search by system only because we formulate searches based on previous results (we are checking now, this could be a bug). Our intent wasn't to try every option, but just code and system|code. We need consistency from servers (not one or the other, depending on server-specific rules that are not discoverable by clients without trial/error or reading documentation). (I previously stated that the guide mixes the two forms, I need to verify that)
Drew Torres (Jun 30 2020 at 17:25):
I do not know a situation where we return system with no code.
Drew Torres (Jun 30 2020 at 17:25):
I am going to double check.
Robert Scanlon (Jun 30 2020 at 17:29):
Looks like it is technically correct, per https://www.hl7.org/fhir/datatypes.html#Coding: If the system is present, and there is no code, then this is understood to mean that there is no suitable code in the system in which to represent the code..
Drew Torres (Jun 30 2020 at 17:29):
Yea it is correct to return a system without a code.
Drew Torres (Jun 30 2020 at 17:30):
The thing I am challenging is the validity of searching by system without codes.
Drew Torres (Jun 30 2020 at 17:34):
It is OK to do that in the spec, but all the example that us-core laid out was using system|code.
Robert Scanlon (Jun 30 2020 at 17:34):
Understood.
Jenni Syed (Jun 30 2020 at 19:13):
As an example of the real world impact here: I search for 123456789, no system... I intend to get SSNs and MRNs and .... on an on that match? And I really don't care which?
Jenni Syed (Jun 30 2020 at 19:13):
Usually even the sys admin use case is "type" and value - which isn't allowed in spec today that I know of :)
Eric Haas (Jun 30 2020 at 19:28):
no intent to search by only system
Eric Haas (Jun 30 2020 at 19:29):
we provide the link to the base spec as helpful information for the reader
Jenni Syed (Jun 30 2020 at 19:33):
@Eric Haas The concern is searching only by code with no system
Jenni Syed (Jun 30 2020 at 19:33):
And the base spec allows for everything, US core doesn't call out what it specifically requires
Eric Haas (Jun 30 2020 at 19:34):
we do in the quick starts.
Eric Haas (Jun 30 2020 at 19:35):
and searching only by code is meant to be supported for the reason discussed above.
Josh Mandel (Jun 30 2020 at 19:35):
Still from a clarity of expectations perspective: "Quick start" doesn't necessarily come across as "complete and authoritative list of requirements for servers".
Josh Mandel (Jun 30 2020 at 19:36):
Something in the quick start I'd assume is definitely allowed, maybe required for servers. Something not in the quick start, I don't know that I'd make any assumptions.
Eric Haas (Jun 30 2020 at 19:36):
it is based on the cap statements and pretty damn thorough
Eric Haas (Jun 30 2020 at 19:37):
at least tried to be
Eric Haas (Jun 30 2020 at 19:37):
and yes if not in there then no assumptions made
Drew Torres (Jun 30 2020 at 19:40):
So http://hl7.org/fhir/us/core/STU3.1/CapabilityStatement-us-core-server.html should be considered authoritative? Then that outlines the exact behavior for token searches per resource.
Drew Torres (Jun 30 2020 at 19:40):
Which I don't object to because the areas of concern like observation.code is explicit in requiring system|code
Drew Torres (Jun 30 2020 at 19:41):
Still a bit of grey area because it is listed under example....
Drew Torres (Jun 30 2020 at 19:41):
SHALL patient+code reference+token GET [base]/Observation?patient=[patient]&code=[system]|[code]
Drew Torres (Jun 30 2020 at 19:41):
http://hl7.org/fhir/us/core/STU3.1/CapabilityStatement-us-core-server.html#observation
Josh Mandel (Jun 30 2020 at 19:42):
Indeed, I don't see a clear statement of whether a server needs to support "Observation?code=123".
Eric Haas (Jun 30 2020 at 19:42):
so one of the dangers of making the capstatement pretty and easy to read is over-interpretation:
Eric Haas (Jun 30 2020 at 19:42):
see the table heading says 'examples'
Eric Haas (Jun 30 2020 at 19:43):
you can't express the nuances of search with the capstatement alone.
Josh Mandel (Jun 30 2020 at 19:43):
Yes. So my question is what is authoritative if these are examples.
Josh Mandel (Jun 30 2020 at 19:43):
Precisely!
Josh Mandel (Jun 30 2020 at 19:43):
so we have a capability statement that doesn't Express the details of what is required (in the computable portions at least it simply can't) and examples that only show a subset of what is required.
Josh Mandel (Jun 30 2020 at 19:44):
Which means it's not obvious whether searching by code without a system is required by this guide.
Josh Mandel (Jun 30 2020 at 19:44):
Unless I'm missing something that clarifies one way or the other.
Eric Haas (Jun 30 2020 at 19:46):
the quick start contain conformance language on purpose:
Eric Haas (Jun 30 2020 at 19:47):
They fill in the details
Eric Haas (Jun 30 2020 at 19:48):
This is nothing new so I am surprised these questions are being asked now....
Josh Mandel (Jun 30 2020 at 19:51):
So it's the curly braces around system that tells you a server must support queries where the client omits a system? I think that's a fine answer, just saying it's not obvious. Is that curly brace thing a FHIR convention?
Josh Mandel (Jun 30 2020 at 19:51):
(and ... people are paying attention now in a deeperway because they are starting to implement against the onc final rule :-))
Eric Haas (Jun 30 2020 at 19:52):
Actually I think that Drew's concern about token search by system only was appropriate
Josh Mandel (Jun 30 2020 at 19:52):
So your reading Eric is that a US Core server must support searching by code, as well as system plus code, but need not support searching by system alone?
Eric Haas (Jun 30 2020 at 19:54):
we discuss the syntax here... but ought reference it in each quick start - ( it that a technical correction or change request? )
Eric Haas (Jun 30 2020 at 19:54):
yes
Josh Mandel (Jun 30 2020 at 19:55):
Content surrounded by {} is optional
Does this mean mandatory for servers to support queries that omit this this content, and optional for clients to supply?
Josh Mandel (Jun 30 2020 at 19:56):
I'm starting to realize how important it is to spell out these distinctions. We've had a number of corrections in the bulk data specification on this kind of issue.
Eric Haas (Jun 30 2020 at 20:12):
Yes I seeing that now... smugly reviewing the syntax I realized that it is slightly inconsistent and not explicitly described.
should be {system|}[code] instead of {[system]}|[code]
and then update the syntax section to call this out and add it to the master example
I think these are technical corrections... but I don't want to mess up the inferno folks here @Robert Scanlon , @Yunwei Wang would this change to the syntax mess things up on your end if the intent was the same. ( it wasn't intended to be computable but just making sure)
Robert Scanlon (Jun 30 2020 at 20:15):
I don't believe we compute off that syntax... where specifically in the artifacts are you going to make that change?
Robert Scanlon (Jun 30 2020 at 20:21):
Inferno's test system is written so both system|code and code are tried for all token search types (if we ever query by just system, its a bug, and we are looking into it).
Robert Scanlon (Jun 30 2020 at 20:24):
Note that there is a similar ambiguity on reference search types, I believe. ?patient=123 vs ?patient=Patient/123.
Yunwei Wang (Jun 30 2020 at 20:26):
I believe it is this one http://hl7.org/fhir/us/core/STU3.1/StructureDefinition-us-core-observation-lab.html#mandatory-search-parameters item 2
code={[system]}|[code],{[system]}|[code] should be code={system|}[code],{system|}[code]
Eric Haas (Jun 30 2020 at 20:27):
proposed changes -
update syntax to clarify token parameters syntax {system|}[code] where the stuff inside the curly braces are optional
and then update all the Quick Starts toconsistently use this syntax like Yunwei posted.
Eric Haas (Jun 30 2020 at 20:28):
If that is heplful can @Yunwei Wang make a tracker?
Eric Haas (Jun 30 2020 at 20:33):
Note that there is a similar ambiguity on reference search types, I believe. ?patient=123 vs ?patient=Patient/123.
Yes the guide is silent on that
Robert Scanlon (Jun 30 2020 at 20:34):
"the stuff inside the curly braces are optional" -- for the server to support or the client to support?
Eric Haas (Jun 30 2020 at 20:42):
Are there client certifications?
Drew Torres (Jun 30 2020 at 20:48):
Just to make sure on this change... server would be expected to support only system|code and code only will be a should?
Robert Scanlon (Jun 30 2020 at 20:48):
The new ONC certification criteria is not for clients.
Josh Mandel (Jun 30 2020 at 20:53):
I heard something different @Drew Torres: servers would be required to support queries with ?code=123 and ?code=http://system.example.org|123
Josh Mandel (Jun 30 2020 at 20:53):
@Eric Haas does that match your expectation?
Drew Torres (Jun 30 2020 at 20:54):
I just reread Eric's comments... should be {system|}[code] instead of {[system]}|[code] system is optional to pass in to the server...
Drew Torres (Jun 30 2020 at 20:55):
Which means we must support a code only search
Josh Mandel (Jun 30 2020 at 20:55):
+1, that's what I'm saying too.
Drew Torres (Jun 30 2020 at 20:57):
I am begrudgingly OK with that. :rolling_on_the_floor_laughing: As long as we get rid of system only.
Brett Marquard (Jun 30 2020 at 21:03):
Definitely meant to support code search
Yunwei Wang (Jun 30 2020 at 21:47):
https://jira.hl7.org/browse/FHIR-27897
Yunwei Wang (Jul 06 2020 at 18:58):
Another question; In CareTeam, Quick Start says: "SHALL support searching using the combination of the patient and category search parameters:
GET [base]/CarePlan?patient=[reference]&category=http://hl7.org/fhir/us/core/CodeSystem/careplan-category|assess-plan"
I understand that server SHALL support category=http://hl7.org/fhir/us/core/CodeSystem/careplan-category|assess-plan. SHOULD server also support category=assess-plan? Since [system] is not optional, I assume the answer is no.
Josh Mandel (Jul 07 2020 at 14:06):
I would have assumed there was a uniform expectation in US Core about variations in a token search need to be supported, irrespective of whether a specific example is provided for each query. I kind of thought our discussion above was about all the places where token search is used. But maybe I overgeneralized?
Last updated: Apr 12 2022 at 19:14 UTC
