FHIR Chat · cql-execution, cql-exec-fhir, cql-exec-vsac questions · cql

Stream: cql

Topic: cql-execution, cql-exec-fhir, cql-exec-vsac questions

Vasyl Herman (Feb 23 2021 at 17:37):

Hi guys! Can you please help me with cql-exec-vsac? I have got an UMLS account from here: my username and API key. I am reading the cql-exec-examples and it says that I need to provide a UMLS_USER_NAME and UMLS_PASSWORD. The question is: does it support API key?

Vasyl Herman (Feb 23 2021 at 17:39):

Sorry, I have found this https://github.com/cqframework/cql-exec-vsac#using-umls-credentials

Chris Moesel (Feb 23 2021 at 17:45):

Hi @Vasyl Herman -- it looks like you found what you need, but let us know if you run into any trouble with it.

Vasyl Herman (Feb 23 2021 at 19:30):

@Chris Moesel Can you please take a look at this example: testArtifact I am trying to use ensureValueSetsWithAPIKey method.

I don't know what is wrong, I am getting an error:

D:\git-cql-fhir\cql-exec-fhir-example>node testArtifact.js
D:\git-cql-fhir\cql-exec-fhir-example\node_modules\cql-exec-vsac\lib\CodeService.js:102
    const filteredVSList = valueSetList.filter(vs => {
                                        ^

TypeError: valueSetList.filter is not a function
    at CodeService.ensureValueSetsWithAPIKey (D:\git-cql-fhir\cql-exec-fhir-example\node_modules\cql-exec-vsac\lib\CodeService.js:102:41)
    at Object.<anonymous> (D:\git-cql-fhir\cql-exec-fhir-example\testArtifact.js:27:13)
    at Module._compile (internal/modules/cjs/loader.js:1063:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
    at Module.load (internal/modules/cjs/loader.js:928:32)
    at Function.Module._load (internal/modules/cjs/loader.js:769:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:72:12)
    at internal/main/run_main_module.js:17:47

to reproduce it:

1. clone
2. npm i
3. gradlew.bat cql2elm
4. node testArtifact.js
   I have committed credential as well :) need to change later ...

Vasyl Herman (Feb 23 2021 at 19:41):

it may happen because I don't have valueset defined in the .cql file :)

Vasyl Herman (Feb 23 2021 at 19:42):

no, the same issue ... with valueset "Clinical Oral Evaluation": 'http://cts.nlm.nih.gov/fhir/ValueSet/2.16.840.1.113883.3.464.1003.125.12.1003'

Vasyl Herman (Feb 23 2021 at 19:48):

ohhh... I should use ensureValueSetsInLibraryWithAPIKey I am sorry about that

Chris Moesel (Feb 23 2021 at 19:53):

Hi @Vasyl Herman -- I don't have time to try running this right now, but looking at the code, I think it is because you are not allowing the code service to fully load the value sets before running the execution. The ensureValueSetsWithAPIKey function returns a Promise -- which means that function is not necessarily _done_ when it returns. In order to ensure that the execution does not happen until the value sets are loaded, you need to invoke the execution only when the promise is resolved (in the then callback). As it is now, you try to execute as soon as the ensureValueSetsWithAPIKey function returns -- which at that point is an unfulfilled promise, so it isn't going to work since the value sets are not yet fully downloaded (although it might work if your cache is fully loaded and complete).

See this block of code from cql-exec-examples -- where we don't actually attempt to execute the CQL until after the code service as ensured the value sets. (Although note that we need to update the examples to use the new API key based function instead).

Vasyl Herman (Feb 23 2021 at 19:54):

Works perfect. I should have been like this:

codeService.ensureValueSetsInLibraryWithAPIKey(library, true, 'f52a90ad-4d3b-4955-ba5c-635264536b53', true)
    .then(() => console.log('Value Sets are downloaded'))
    .catch((err) => console.log('Error downloading value sets', err))

Vasyl Herman (Feb 23 2021 at 19:55):

@Chris Moesel Thanks!

Chris Moesel (Feb 23 2021 at 19:56):

I think my suggestion above still stands (regarding the Promise). My guess is that if you delete your value set cache files, it will fail again (at least the first time you run it). To avoid this, you need to properly wait for that promise to be fulfilled.

Vasyl Herman (Feb 23 2021 at 19:56):

Right, I agree. I will fix it later

Vasyl Herman (Feb 23 2021 at 20:10):

BTW, ensureValueSetsInLibraryWithAPIKey seems not to be described here: cql-exec-vsac. Let me ask you, what is the difference ensureValueSetsWithAPIKey and ensureValueSetsInLibraryWithAPIKey
I am reading this but it's not clear to me.
image.png
I am promising to contribute into the README.md to clarify this aspect :)

Vasyl Herman (Feb 24 2021 at 18:12):

Hello,
Could you please help with the libraries:

const elmFile = JSON.parse(fs.readFileSync(
    path.join(__dirname, 'r4', 'cql', 'exm74', 'EXM74-10.2.000.json'),'utf8'));

const libraries = {
    FHIRHelpers: JSON.parse(fs.readFileSync(path.join(__dirname, 'r4', 'cql', 'exm74', 'FHIRHelpers-4.0.1.json'), 'utf8')),
    Global: JSON.parse(fs.readFileSync(path.join(__dirname, 'r4', 'cql', 'exm74', 'MATGlobalCommonFunctions-5.0.000.json'), 'utf8')),
    Hospice: JSON.parse(fs.readFileSync(path.join(__dirname, 'r4', 'cql', 'exm74', 'Hospice-2.0.000.json'), 'utf8')),
    SDE: JSON.parse(fs.readFileSync(path.join(__dirname, 'r4', 'cql', 'exm74', 'SupplementalDataElements-2.0.0.json'), 'utf8')),
};
const library = new cql.Library(elmFile, new cql.Repository(libraries));

Is it necessary to manually define all the libraries that EXM74-10.2.000.json includes? How can I manage this in more elegant way?

• VH

Chris Moesel (Feb 24 2021 at 18:45):

I suppose there are a couple of things you could try to avoid having to build up that JSON libraries object. One would be to write some code that reads a folder, enumerates its files, detects which ones are valid ELM JSON files, and creates the JSON object containing all those libraries programatically. Then you can point it at any folder of CQL and say "go" (although you might need to exclude the main library from that object).

Another is that you could write your own implementation of the cql.Repository that gets passed into the cql.Library constructor. I think that any instance that has a resolve(library, version) function which returns an instance of a Library should work. So you could write your own implementation that resolves libraries from a file system, a server, etc.

Of course, both those solutions require more up front work to write the code that does that stuff -- but once you've done it, then loading arbitrary libraries and their dependencies would be easier.

Vasyl Herman (Mar 12 2021 at 06:48):

Hello,

We have got a CQL library that uses NCQA HEDIS value sets:

Value sets in this library included per NCQA HEDIS Specifications MY 2020 Volume 2 Value Set Directory 2020-11-13
Value sets in this library have not been added to VSAC

I was wondering whether the cql-execution is able to parse Value Set Directory (VSD - an Excel file) as they currently only provide the value sets in Excel.
We have to parse the Excel file into value set objects some how.

Please, we would appreciate any help on this question.

Chris Moesel (Mar 12 2021 at 13:29):

Hi @Vasyl Herman. We don't currently have any support for VSD files.

The cql-execution library includes a very simple CodeService that accepts a JSON-formatted map of value sets (by OID and then version). You can see an example of the JSON it accepts in the CodeService tests. One approach to supporting VSD files would be to write a script that can parse them into the JSON format that the CodeService expects.

Another approach would be to write your own CodeService implementation (like cql-exec-vsac) that knows how to read VSD files. This approach might be more efficient (depending on implementation) but is likely more work.

Bryn Rhodes (Mar 12 2021 at 18:19):

There's tooling here that might be able to help here, takes ValueSet resources and turns them into a ValueSet-db.json file that works with the JavaScript CQL engine: https://github.com/cqframework/cqf-tooling/blob/master/src/main/java/org/opencds/cqf/tooling/terminology/ToJsonValueSetDbOperation.java

Bryn Rhodes (Mar 12 2021 at 18:19):

There's also tooling there to process spreadsheets in various formats to produce ValueSet resources.

Bryn Rhodes (Mar 12 2021 at 18:19):

So you could put together a process chain that works out of the components there.

Vasyl Herman (Mar 15 2021 at 08:39):

Hello, Thank you for valuable information!!!
I was wondering if it's possible to build a gradle script with VsacXlsxToValueSet task like cql2elm task
Thanks!

Bryn Rhodes (Mar 15 2021 at 13:33):

It should be yes, though I've never built a gradle task to execute a command-line. We typically use bash or command scripts when putting things like that together.

Chris Moesel (Mar 15 2021 at 13:42):

I'm not familiar enough w/ the tools Bryn noted to put together a gradle script for them, but the overall approach to calling a Java CLI in gradle is fairly straight-forward. Here is a script I use in my artifact development repositories:

plugins {
  id 'java'
}

repositories {
  mavenCentral()
}

dependencies {
  runtimeOnly 'info.cqframework:cql-to-elm:1.5.1'
}

task cql2elm(type: JavaExec) {
  classpath = sourceSets.main.runtimeClasspath
  main = 'org.cqframework.cql.cql2elm.CqlTranslator'
  args '--input', './src/cql', '--format', 'JSON'
}

The nice thing about this approach is that you can easily switch to a different version of cql-to-elm by just modifying the dependency version. You can also override the dependency with a local version by putting this in your settings.gradle file:

includeBuild "/your/path/to/cqframework/clinical_quality_language/Src/java"

Chris Moesel (Mar 15 2021 at 13:43):

Oh. Whoops. I just noticed that this is the same basic script @Basil German already linked to above. Oh well. I guess it saves people a click.

Bryn Rhodes (Mar 15 2021 at 14:48):

And here are some example scripts for this type of processing chain we've done:

java -jar CQFTooling.jar -VsacXlsxToValueSetBatch -ptsd="Src\YourProject\input\vocabulary\valueset\spreadsheets" -burl="http://cts.nlm.nih.gov/fhir/ValueSet/"

java -jar CQFTooling.jar -EnsureExecutableValueSet -path="Src\YourProject\input\vocabulary\valueset"

java -jar CQFTooling.jar -ToJsonValueSetDb -path="Src\YourProject\output"

Vasyl Herman (Mar 15 2021 at 16:21):

Thanks, Bryn!
Thanks, Chris!
I will try it. :+1:

Vasyl Herman (Mar 15 2021 at 17:57):

Could you please help me with that?
I am trying to build a gradle task to to convert HEDIS VSD to valusets. I am not sure if VsacXlsxToValueSetBatch is able to convert VSD as it might be different from VSAC xlsx file.
What about if I compare VSAC and VSD files and bring VSD file to the structure as VSAC xlsx file is?
Does it make sense to do?

HEDIS Value Set Directory (VSD) is included in this repo

The task I am trying to build:

plugins {
  id 'java'
}

repositories {
  mavenCentral()
}

dependencies {
  runtimeOnly 'org.opencds.cqf:tooling:1.3.0'
}

task xlsxToValueSet(type: JavaExec) {
  classpath = sourceSets.main.runtimeClasspath
  main = 'org.opencds.cqf.tooling.Main'
  args '-VsacXlsxToValueSet', '-pts=./HEDIS-MY-2020-Volume-2-Value-Set-Directory-2020-11-13.xlsx'
}

Thanks!

Vasyl Herman (Mar 15 2021 at 18:20):

Also I have found that there is -XlsxToValueSet option in Main class
But it doesn't gives any examples:

        - Generic Excel spreadsheet to FHIR ValueSet resource conversion
            - command: mvn exec:java -Dexec.args="[-XlsxToValueSet] [-pathtospreadsheet | -pts] (-outputpath | -op) (-encoding | -e)"
            - Example: TODO
            - This tooling converts an Excel spreadsheet (.xlsx extension) to a FHIR ValueSet resource
            - This is highly configurable
                - TODO

So, I am trying java -jar tooling-1.3.0-jar-with-dependencies.jar -XlsxToValueSet -pts="HEDIS-MY-2020-Volume-2-Value-Set-Directory-2020-11-13.xlsx" with an error:

Exception in thread "main" java.lang.IllegalArgumentException: -code flag must be specified
        at org.opencds.cqf.tooling.terminology.GenericValueSetGenerator.execute(GenericValueSetGenerator.java:253)
        at org.opencds.cqf.tooling.Main.main(Main.java:159)

I appreciate any help!

Vasyl Herman (Mar 15 2021 at 18:21):

Link to HEDIS VSD on Google Drive

Vasyl Herman (Mar 16 2021 at 11:44):

Hello, all!
So, I have script and seems like it's working:

plugins {
  id 'java'
}
repositories {
  mavenCentral()
}
dependencies {
  runtimeOnly 'org.opencds.cqf:tooling:1.3.0'
}

task hedisXlsxToValueSet(type: JavaExec) {
  classpath = sourceSets.main.runtimeClasspath
  main = 'org.opencds.cqf.tooling.Main'
  args '-HedisXlsxToValueSet', '-pts=./HEDIS-MY-2020-Volume-2-Value-Set-Directory-2020-11-13.xlsx', '-op=./vsd'
}

It generates a bunch of JSONs in vsd directory.

like so:

{
  "resourceType": "ValueSet",
  "id": "2.16.840.1.113883.3.464.1004.1509",
  "url": "http://ncqa.org/fhir/hedis/ValueSet/2.16.840.1.113883.3.464.1004.1509",
  "identifier": [ {
    "system": "urn:ietf:rfc:3986",
    "value": "2.16.840.1.113883.3.464.1004.1509"
  } ],
  "version": "2020-10-01",
  "name": "37WeeksGestation",
  "title": "37 Weeks Gestation",
  "status": "active",
  "publisher": "National Committee for Quality Assurance (NCQA)",
  "compose": {
    "include": [ {
      "system": "http://hl7.org/fhir/sid/icd-10-cm",
      "version": "2021.1.20AA",
      "concept": [ {
        "code": "Z3A.37",
        "display": "[Z3A.37] 37 weeks gestation of pregnancy"
      } ]
    }, {
      "system": "http://snomed.info/sct",
      "version": "2020.03.19AB",
      "concept": [ {
        "code": "43697006",
        "display": "Gestation period, 37 weeks (finding)"
      } ]
    } ]
  }
}

Vasyl Herman (Mar 16 2021 at 15:11):

Could anyone, please, give me an idea how to :cookie: feed up cql-execution library by Value Set objects like that one above

Bryn Rhodes (Mar 16 2021 at 15:18):

Once you have those ValueSets, those are "computable" (i.e. defined with a compose and need to be "expanded" before they can be used). So there is tooling to do that, the "EnsureExecutableValueSet" operation:

java -jar CQFTooling.jar -EnsureExecutableValueSet -path="Src\YourProject\input\vocabulary\valueset"

Once you have those "executable" value sets, you can then use the "ToJsonValueSetDb" to produce the valueset-db.json file that the JavaScript engine expects:

java -jar CQFTooling.jar -ToJsonValueSetDb -path="Src\YourProject\output"

Put the output of that in the cql path for the JavaScript engine, and that should do it.

Vasyl Herman (Mar 16 2021 at 15:31):

Thanks! @Bryn Rhodes
Seems like I'm looking in a wrong place. There is no operation like EnsureExecutableValueSet or ToJsonValueSetDb in Main
the command:

java -jar tooling-1.3.0-jar-with-dependencies.jar -EnsureExecutableValueSet -path="vsd"

as weel ToJsonValueSetDb as gives me

D:\git-cql-fhir\cql-exec-fhir-example>java -jar tooling-1.3.0-jar-with-dependencies.jar -EnsureExecutableValueSet -path="vsd"
Exception in thread "main" java.lang.IllegalArgumentException: Invalid operation: EnsureExecutableValueSet
        at org.opencds.cqf.tooling.OperationFactory.createOperation(OperationFactory.java:84)
        at org.opencds.cqf.tooling.Main.main(Main.java:159)

Dave Carlson (Mar 16 2021 at 15:38):

@Vasil Herman These commands work for me in a similar use case, but I am using tool-1.3.1-jar. Try updating your build of the tooling.

Bryn Rhodes (Mar 16 2021 at 15:39):

Ah yes, use the 1.3.1-SNAPSHOT version...

Vasyl Herman (Mar 16 2021 at 15:57):

amazing finally I found 1.3.1
Thanks!

Vasyl Herman (Mar 16 2021 at 18:33):

Thanks!

plugins {
  id 'java'
}

repositories {
  maven {url 'https://oss.sonatype.org/content/repositories/snapshots'}
}

dependencies {
  runtimeOnly 'org.opencds.cqf:tooling:1.3.1-SNAPSHOT'
}

task hedisXlsxToValueSet(type: JavaExec) {
  classpath = sourceSets.main.runtimeClasspath
  main = 'org.opencds.cqf.tooling.Main'
  args '-HedisXlsxToValueSet', '-pts=./HEDIS-MY-2020-Volume-2-Value-Set-Directory-2020-11-13.xlsx', '-op=./vsd'
}

task ensureExecutableValueSet(type: JavaExec) {
  classpath = sourceSets.main.runtimeClasspath
  main = 'org.opencds.cqf.tooling.Main'
  args '-EnsureExecutableValueSet', '-path=./vsd'
}

task toJsonValueSetDb(type: JavaExec) {
  classpath = sourceSets.main.runtimeClasspath
  main = 'org.opencds.cqf.tooling.Main'
  args '-ToJsonValueSetDb', '-valuesetpath=./vsd', '-outputpath=./vsd-db'
}

It may save someone time

Vasyl Herman (Mar 17 2021 at 16:22):

@Chris Moesel
Hello, is there an example using multiple CodeServices
My cql Library uses multiple Value Sets from HedisXlsxToValueSet and VSAC. I am having two CodeService Objects: they looks fine but I don't know how to put things togather:

        // preparing Hedis VS if exists
        let hedisVSets = {};
        try {
            hedisVSets = JSON.parse(fs.readFileSync(path.join(measurePath, 'hedis_cache', 'valueset-db.json'), 'utf8'))
        } catch (err) {}
        const hedisCodeService = new cql.CodeService(hedisVSets);

        // preparing NLM VS if exists
        let vsacCodeService = new cqlvsac.CodeService(
            path.join(measurePath, 'vsac_cache'),true);

        Object.assign(vsacCodeService.valueSets, hedisCodeService.valueSets)

        try {
            await vsacCodeService.ensureValueSetsInLibraryWithAPIKey(library, true , undefined, true)

        } catch (err){
            console.log(`can't download`)
        }

Chris Moesel (Mar 17 2021 at 19:55):

I think the best approach would be to create a new class that can contain the other two CodeService instances. That new class needs to provide the same basic interface (w/ functions for findValueSetsByOID(oid) and findValueSet(oid, version)) -- and its implementation could invoke each contained service, returning the results of whichever one is able to actually resolve the VS.

Vasyl Herman (Mar 18 2021 at 06:54):

Sounds reasonable!
But what about vsacCodeService.ensureValueSetsInLibraryWithAPIKey? I still would like to download VSs from VSAC and I see attempts to download VSs form NCQA as well even though it already present in vsacCodeService.valueSets
image.png
image.png
image.png

I suppose that only VS that are absent in a CodeService.valueSets instance should be downloaded, right? Or it tries to download the latest version? Or it has to be present in library.valueSets.
I am not sure. Please, let me know.

Vasyl Herman (Mar 18 2021 at 07:42):

The question is how can I cheat on the vsac library so it downloads missing VSs only.
What Object is responsible for storing present VSs?

Vasyl Herman (Mar 18 2021 at 11:47):

Never mind, sorry guys! I have found a typo. the code above works fine!

Vasyl Herman (Mar 18 2021 at 11:56):

Chris Moesel, it's not necessary to create a new class, I feel like Object.assign() is a good fit here. will see...

Vasyl Herman (Apr 13 2021 at 18:39):

Hi, @Chris Moesel !
I was wondering if there is a way to set default Measurement Period "on the fly" at execution time?
I mean if someone doesn't want to set default Interval in cql file like:

parameter "Measurement Period" Interval<DateTime>
    default Interval[@2010-01-01T00:00:00.0, @2020-01-01T00:00:00.0) // here

instead it would be (set up dynamically depending on current year) in index.js like:

library.parameters.magicMethotToSetDefInterval(2019-01-01T00:00:00.0, 2020-01-01T00:00:00.0)

Something like that...
Thanks!

Chris Moesel (Apr 13 2021 at 18:51):

Hi @Vasil Herman! The spec doesn't define anything like that. If you wanted to do something like that, I think you'd need to implement something to get that "on the fly" value from wherever you want to get it and then pass it in as a parameter value to the engine when you invoke it.

Last updated: Apr 12 2022 at 19:14 UTC