Stream: hapi
Topic: Subscription matching
Seth Rylan Gainey (Apr 10 2020 at 19:54):
Our team is implementing Subscriptions using HAPI 3.8, but the limit of 1000 active subscriptions is a downside. I see that MAX_SUBSCRIPTION_RESULTS was increased to 50k in 4.1, but it's not clear why this limit was lifted. Is there a performance improvement with subscription matching in 4.1?
James Agnew (Apr 10 2020 at 22:00):
Hi Seth- Yup, in 4.1 we added "in memory matching", which allows many subscriptions (depending on the complexity of the criteria you use) to be processed without any database interaction at all. This dramatically increases the scalability of the subscription matcher.
When you submit a subscription to the server, you'll notice an extension added to the response. It details whether your criteria is able to take advantage of the in-memory matcher.
Seth Rylan Gainey (Apr 13 2020 at 12:57):
The changelog and javadocs say that in-memory matching was in 3.7+
1.1.7HAPI FHIR 3.7.0 (Gale)
Changed subscription processing, if the subscription criteria are straightforward (i.e., no chained references, qualifiers or prefixes) then attempt to match the incoming resource against the criteria in-memory
Although there as an improvement added in 4.1.0
Add support for in-memory matching on date comparisons ge,gt,eq,lt,le.
We'll continue testing. Does it sound right that the upper bound for active Subscription with "non-in-memory" criteria is 1000, and about 50k for in-memory criteria? If so, we may just look at disabling the use of non-in-memory criteria for our server, since they would be a small set of Subscriptions, but a disproportionate performance cost.
James Agnew (Apr 16 2020 at 15:53):
Hmm, yeah I guess my timeline was wrong. That sounds correct to me.
Last updated: Apr 12 2022 at 19:14 UTC
