Stream: hapi
Topic: asynchrounous search indexes
Job Schipper (Oct 19 2020 at 13:51):
Hi,
From what I can tell, it looks like the search indexing after i.e. creating a new Patient resource is being done asynchronously. If i directly query for my newly created patient based on a ?identifier= it will not find it.
Is this behavior configurable? Is there for instance an option to have the POST request wait with returning until the indexes are updated?
James Agnew (Oct 19 2020 at 14:19):
Not currently possible - The assumption is that reindexing is going to take a very long time on real production systems with ream amounts of data, so a blocking query wouldn't really be feasible.
Job Schipper (Oct 22 2020 at 11:41):
Are there any fields other than the logical id field that are highly consistent when you do a read/search request? I mean fields that are indexes instantly pas part of i.e. an update operation.
Job Schipper (Oct 22 2020 at 12:20):
for instance the _tag field? or is that also part of the async index update?
James Agnew (Oct 22 2020 at 12:43):
I'm not sure I understand what you mean by that
Job Schipper (Oct 22 2020 at 13:40):
consider this example:
if i create a patient resource with a POST /Patient, i get an id back in the response, which i can use to lookup that patient resource from that point on. However, if i search for it by lastname, it will not find it until the re-indexing has completed.
I was wondering if there are any other fields that i can use to request my just-created Patient resource that do not depend on the re-index
Job Schipper (Oct 22 2020 at 13:41):
or is that limited to the logical id field?
James Agnew (Oct 22 2020 at 21:27):
Well, lastname is a built in search parameter so you should be able to use it right away, unless you had previously removed it and performed a reindex, and you are now re-enabling it.
In other words, reindexing doesn't disable existing search parameters, it only adds new ones to the mix once it's complete.
James Agnew (Oct 22 2020 at 21:28):
oh wait sorry, lastname isn't built in.. family is.
James Agnew (Oct 22 2020 at 21:28):
Point stands though- Any search parameter that was there before you added the new one and started the reindex will work during the reindex too. Including _id and family
Job Schipper (Oct 23 2020 at 11:14):
@James Agnew but if it's a CREATE action, none of the parameters were there before (for this Resource) right? so that means that after a create action you always have to wait until re-indexing is finished before being able to find it? (this is what I seem to be experiencing)
edit: and by "being able to find it" i mean a search call based on i.e. ?identifier= or ?_tag= or ?family=
James Agnew (Oct 23 2020 at 14:40):
I'm sorry, I still don't understand what you are describing. Perhaps you could describe in more detail the steps you are taking that aren't prodicing the expected results?
My only thought here is maybe you are simply getting hit by the query cache: https://smilecdr.com/docs/fhir_repository/performance_and_caching.html
Job Schipper (Oct 23 2020 at 15:52):
@James Agnew ha indeed! I just came to the same conclusion, and this is indeed the issue i was having.
our application that creates fhir resources based on a patient DB does a lookup based on _tag to see if there is an existing Resource for that Patient. When processing an update for a just-created patient, it would hit cache when trying to look it up, thus assuming it does not exist yet. Now using no-cache headers for this specific scenario and all is good!
Thanks again!
Last updated: Apr 12 2022 at 19:14 UTC
