Stream: implementers
Topic: number of resources in FHIR response
Lital Inghel (Dec 21 2021 at 10:18):
does FHIR IG talks about number of resources that should be part of a response? i mean do we must support count, or paging?
René Spronk (Dec 21 2021 at 10:35):
http://build.fhir.org/search.html#count is pretty clear on this: servers don't have to support paging, but if they support any search capabilities at all, they SHALL support _count and return the specified max amount of responses.
Lloyd McKenzie (Dec 21 2021 at 14:35):
More specifically, return no more than the specified max amount of 'matching' resources. They can return less. And _count doesn't control the number of _include and _revinclude resources.
Lital Inghel (Dec 21 2021 at 14:47):
so _count is a shall? couldn't see that in the IG
René Spronk (Dec 21 2021 at 14:48):
It states "NOTE: This means that all servers that support search or history SHALL support checking the _count parameter. "
Lital Inghel (Jan 10 2022 at 21:18):
interesting, I think in R4 (which is the version I'm working on) it has different wording
image.png
John Moehrke (Mar 23 2022 at 15:08):
does _count specify the max total or the max of the primary search resources? Meaning if I query on X and _include X.references to Y... the resulting bundle will include both X and Y resources. Is _count a limit on X or the limit on X plus Y?
Paul Church (Mar 23 2022 at 15:09):
just X
John Moehrke (Mar 23 2022 at 15:09):
I like that answer, but am not sure everyone sees it that way.
Paul Church (Mar 23 2022 at 15:11):
there is no ambiguity, "In the same way, the _count parameter only applies to resources with entry.search.mode = search, and does not include included resources or operation outcomes." from http://hl7.org/fhir/search.html#count
John Moehrke (Mar 23 2022 at 15:13):
ah, the critical statement
... and does not include included resources or operation outcomes.
John Moehrke (Mar 23 2022 at 15:14):
excellent. this is what I was hoping, as it assures paging is by full counts of the primary thing I am searching on.
Lloyd McKenzie (Mar 23 2022 at 16:36):
It makes _revinclude super dangerous. A _count of 100 can still result in a Bundle of 10k+
John Moehrke (Mar 23 2022 at 16:49):
understood.
Paul Church (Mar 23 2022 at 17:09):
yeah revinclude is very tricky - the Google implementation retrieves one page of 100 for each revinclude clause, and then stops because you have to draw the line somewhere and there's no non-arbitrary way to choose
Elliot Silver (Mar 23 2022 at 19:28):
I suppose you could return fewer primary resources than count, to account for a very large number of secondary resources, but that only means it works until you run into that one primary resource with thousands of revinclude secondary resources.
John Moehrke (Mar 23 2022 at 20:18):
that is what I was trying to make very clear. systems that return less than _count of the primary will make paging not functional.
Michele Mottini (Mar 23 2022 at 21:48):
Return less that _count is ok, it is explicitly called out in the specs '...but may return less than the client requested.'
Paul Church (Mar 24 2022 at 00:55):
Returning less than _count won't break pagination - the client has to trust that the next link provided by the server is appropriate for the actual page size.
John Moehrke (Mar 24 2022 at 12:59):
well, if the UI is fixed at 10 entries, and you return 9... that does, in my view, break pagination. I asked for 10 because I have a UI that shows 10.
John Moehrke (Mar 24 2022 at 13:00):
The _count is the way I can tell the server that I really want 10... I get that there can be exceptions, but I think it is better to put the hard-problem on the server.
John Moehrke (Mar 24 2022 at 13:00):
an ask for _count of 10 really should be strongly urged to return 10 (except, clearly, for the last page).
Craig McClendon (Mar 24 2022 at 14:02):
I doubt many implementations will arbitrarily return you 9 resources when you request 10, but many may cap you at say 100 were you to request _count=1000000. The server needs to protect itself from overruns.
René Spronk (Mar 24 2022 at 14:37):
Correct. In practice the returned page size is max(client-side _count, server-side page max size). That's also what the FHIR spec defines.
If as a client you assume that a server will always return 10 hits, you'll be in for a surprise..
John Moehrke (Mar 24 2022 at 16:18):
not saying that exceptions won't be appropriately handled.. just wanted to make clear that the requested _count is first the count of primary resources (vs requested _include/_revinclude additions), and second that the server is expected to fill out _count sized pages when possible. - if these were not true, then I would question why we have a _count parameter at all.
Elliot Silver (Mar 24 2022 at 16:21):
René Spronk said:
Correct. In practice the returned page size is max(client-side _count, server-side page max size). That's also what the FHIR spec defines.
Isn't that min(client-side _count, server-side page max size). ?
John Moehrke (Mar 24 2022 at 16:26):
the number of Resources in a search set can be larger than _count... in that the primary resources would not be more than _count.
John Moehrke (Mar 24 2022 at 16:26):
I don't think there is anything about the min.
Elliot Silver (Mar 24 2022 at 16:27):
I meant the number of primary resources returned is the smallest of of client-side count, server-side page max, and actual number of matches.
Lloyd McKenzie (Mar 24 2022 at 16:31):
_count says "don't give me more than this". The server is always allowed to send you less, and is under no obligation to try to provide the number you asked for.
John Silva (Mar 24 2022 at 17:45):
An example for sending less that client-requested _count ---- Imagine the result set in the bundle was a set of VERY large Binary resources; the server may choose to optimize the number of these it sends to minimize it's internal memory footprint, and/or HTTP transfer payload, etc.
Last updated: Apr 12 2022 at 19:14 UTC
