FHIR Chat · Looping through observations

Stream: cql

Topic: Looping through observations

Marisa Hoenig (Jul 27 2021 at 22:59):

We are trying to optimize our CQL, which uses the JS engine. We are currently grabbing only the Heart rate observations from a list of ALL the patient observations using this syntax: [Observation: CC."Heart rate"]. Does anyone know if this syntax loops through all the observations one by one to pick out the heart rates and create a new list? Or is it using a hashmap or some other way to more efficiently grab the selected observations?

Here is our full function, which has helper functions making sure the observation is verified and is within our lookback period:

define "Heart Rate Observations":
  C3F.ObservationLookBack(
    C3F.Verified([Observation: CC."Heart rate"]),
    VitalSignsLookbackPeriod)

Chris Moesel (Jul 28 2021 at 12:53):

Hi @Marisa Hoenig. The JavaScript engine uses the Array.filter javascript function to find the matching records from the full set of records. In fact, first it filters on all the resources in the bundle looking for a match on the dataype (e.g., Observation) and then it filters on the results of that to find matches based on the code or value set (e.g., "Heart rate"). I would guess that most JavaScript engines implement Array.filter as a loop, so yes, it is looping through.

TBH, I was not surprised it was using Array.filter to match on the codes, but I was surprised it was using Array.filter to match on the datatype. I actually assumed it was using a map there (and would like to ask the several-years-ago me why I did not use a map!). So, there is probably some room for improvement (and we happily accept pull requests).

Relevant code:

Marisa Hoenig (Jul 28 2021 at 15:22):

Hey @Chris Moesel ! Thank you so much for all the info. We are taking a look at the code now.

In terms of filtering on datatype, does this happen no matter what, every time you are searching through a bundle? For example, in our use case, our bundle is purely Observations, so we don't need to loop through for datatype — only for codes.

Chris Moesel (Jul 28 2021 at 15:34):

Hi @Marisa Hoenig. Yes, I think it's going to attempt to filter by datatype every time, even if your bundle has all the same datatype. If that bit of code (in cql-exec-fhir) stored the bundle entries as a map (w/ resourceType as key), then it would prevent that repetitive looping. That kind of change seems feasible, but I'm not sure we'll have a chance to really look at it until September. My team's August schedule is very tight. I also don't know how much of a real impact that will have; it probably depends on the number of resources and how often the CQL is calling into that (i.e., how many retrieves you have). The only way to guess the impact would be to try it (or do some performance profiling on the existing code).

Marisa Hoenig (Jul 29 2021 at 22:09):

Thanks for all the help, @Chris Moesel! We are looking more closely at our CQL and seeing where we can reduce the number of times we're looping through all observations. We'll see if there's any spot for us to improve the code and submit a PR. We've done some initial profiling of the CQL to identify where the problem is, so hopefully we can resolve it soon.

Last updated: Apr 12 2022 at 19:14 UTC

Main menu