FHIR Chat · Including binary CQL · IG creation

Stream: IG creation

Topic: Including binary CQL

Noemi Deppenwiese (Apr 22 2020 at 14:40):

I recently found out that the Publisher allows storing base64-encoded CQL in seperate files (nice!). But i tried to use this functionality in I got an error:

Translating CQL source                                                           (00:27.0066)
Translating CQL source in folder D:\Uni\bbmri-fhir-ig\input\other                (00:27.0067)
Translating CQL source in file D:\Uni\bbmri-fhir-ig\input\other\test.cql         (00:27.0075)
Translation failed with (1) errors; see the error log for more information.      (00:27.0706)
Publishing Content Failed: Error doing replacements - no id defined in resource: Bundle/null (00:28.0184)

In the Library i have

 "content": [
    {
      "id": "ig-loader-test.cql"
    }
  ]

So it seems to find the file, but somehow needs another id? I understood that test.cql only needs to include the base64-encoded CQL, or does it need a wrapper of some sorts?

Bob Dolin (Apr 22 2020 at 17:14):

@Yan Heras and I were also trying to figure out what to do here. Last week, we grabbed the sample-ig and did a build, but we don't see the CQL showing up anywhere, and I'm not sure if it's something we're doing wrong or an issue with the build process. Can anyone offer advice?

Bryn Rhodes (Apr 22 2020 at 18:04):

Yes, the library resource is a skeleton for what is published in the output. It takes source CQL files included in the input/cql directory (or whatever directory is configured in that parameter in the IG), performs the translation and packages the CQL content in the publisher library in the output.

Bryn Rhodes (Apr 22 2020 at 18:06):

So this example.cql for example:

Bryn Rhodes (Apr 22 2020 at 18:06):

https://github.com/FHIR/sample-ig/blob/master/input/cql/example.cql

Bryn Rhodes (Apr 22 2020 at 18:07):

Is referenced by the id 'ig-loader-example.cql' from this library resource: https://github.com/FHIR/sample-ig/blob/master/input/examples/Library-example.json

Bob Dolin (Apr 22 2020 at 18:38):

Thanks @Bryn Rhodes . We started with the sample-ig, and replicated the approach, but we must be doing something wrong since we're not seeing the CQL. Might be better to wait until we move our draft IG over to the HL7 build site, so it's easier to see what we've done. So @Yan Heras , maybe we should just study this a bit more, and then revisit once we're up on HL7 build site?

Yan Heras (Apr 22 2020 at 19:11):

Hi @Bryn Rhodes, I see the publisher has put the cql content in the output file "Library-example.json.html" in content with contentType "text/cql", however, is the publisher able to output the actual source cql (not after the translation) ? something like what's in the Opioid IG where you can see the author view of the cql file (http://build.fhir.org/ig/cqframework/opioid-cds/Library-omtklogic.html).

Bryn Rhodes (Apr 22 2020 at 19:22):

We are working to add a template for the Library that will include the CQL as part of the generated Narrative, something like this: http://build.fhir.org/ig/cqframework/opioid-cds/Library-opioidcds-recommendation-10.html

Yan Heras (Apr 22 2020 at 19:35):

Great, thank you! is there a time frame for this new template to be available?

Bryn Rhodes (Apr 22 2020 at 19:55):

Should be soon, I'll be working on it tonight.

Yan Heras (Apr 22 2020 at 20:17):

Thanks Bryn!

Bob Dolin (Sep 17 2020 at 17:27):

Hi @Bryn Rhodes . Sorry if this is a naive question, since I'm still a bit new to the IG publication process. In the draft Outcome Criteria Framework IG [http://build.fhir.org/ig/HL7/fhir-outcome-criteria-framework-ig/branches/master/index.html], I've created a FHIR Library profile that says that Library.content must reference a CQL file (rather than include it in Library.content.data), but the publishing process automatically pulls the CQL into Library.content.data. Is there a way to avoid having the publisher pull the CQL into Library.content.data, and instead point to the CQL file so that a reader can download it to view it?

Bryn Rhodes (Sep 17 2020 at 22:44):

Well, you could, but there's a whole bunch of extra processing that "pulling the CQL into the Library" does that is important for packaging the CQL library as a FHIR Library resource. Things like automatically exposing data requirements, parameters, and related artifacts. You'd have to do all of that another way if you didn't allow the publisher to do it's thing there.

Grahame Grieve (Sep 17 2020 at 22:51):

why not include it in the data?

Bob Dolin (Sep 18 2020 at 01:02):

Somehow I thought content had to be either referenced or directly embedded, but it looks like I was mistaken, so pulling the CQL into the LIbrary doesn't hurt anything. Is there a way to also point to the CQL files themselves, so readers can see them in text format?

Grahame Grieve (Sep 18 2020 at 01:38):

hmm I thought that happened already

Bryn Rhodes (Sep 18 2020 at 03:42):

The Library template includes the CQL as part of the fenerated narrative, as well as an anchor so you can point to it directly.

Bryn Rhodes (Sep 18 2020 at 03:43):

*generated

Grahame Grieve (Sep 18 2020 at 03:43):

I prefer fenerated. or maybe fenestrated

Bryn Rhodes (Sep 18 2020 at 03:44):

lol

Bob Dolin (Sep 18 2020 at 14:06):

Sorry @Bryn Rhodes , @Grahame Grieve I'm not following you. You are saying that somewhere in the published output there is a way to see the CQL file itself, so that a reviewer of the IG can just click it and see the text contents?

Bryn Rhodes (Sep 18 2020 at 14:11):

No, I'm saying that the CQL is in the generated narrative: http://build.fhir.org/ig/cqframework/opioid-cds-r4/Library-library-OpioidCDSREC01.html

Bob Dolin (Sep 18 2020 at 14:23):

Yes, that's exactly what I'm looking for, but I don't know how to get it to generate. Is there something I need to include in the Library to cause that to generate? When I upload a Library example, I include this: "content": [{"id": "ig-loader-DepressionOutcomes.cql"}. The output I get is just the base64 encoded CQL showing up in the data field.

Grahame Grieve (Sep 18 2020 at 19:56):

have you got an example you can point to?

Bryn Rhodes (Sep 18 2020 at 20:03):

It's a template that you need to include. We're working on getting it to the base template but I don't think it's there yet (or it would already be happening, right). https://github.com/FHIR/sample-ig/blob/master/templates/liquid/Library.liquid

Bryn Rhodes (Sep 18 2020 at 20:08):

Put that template in that folder, and put this parameter in your IG:

Bryn Rhodes (Sep 18 2020 at 20:08):

https://github.com/FHIR/sample-ig/blob/master/input/myig.xml#L129

Bob Dolin (Sep 18 2020 at 20:18):

@Grahame Grieve This is for the draft Outcome Criteria Framework IG [http://build.fhir.org/ig/HL7/fhir-outcome-criteria-framework-ig/index.html], where I'm trying to introduce a FHIR Library profile that requires a pointer to a CQL file, and that also publishes such that viewers can see the text of the CQL file.

Bob Dolin (Sep 18 2020 at 20:19):

@Bryn Rhodes can you tell me what parameter code to use?

Bryn Rhodes (Sep 18 2020 at 20:21):

Include this parameter:

    <parameter>
      <code value="path-liquid"/>
      <value value="templates\liquid"/>
    </parameter>

That parameter is a relative path to the templates folder, where the templates are loaded by resource name. There's documentation on implementation guide parameters here for more information too.

Grahame Grieve (Sep 18 2020 at 20:23):

@Lloyd McKenzie the template is rendering this as an example, not a canonical resource. Why?

Lloyd McKenzie (Sep 18 2020 at 20:37):

The example layout is the default layout - I presume we don't have a layout specified for Library. What would we want such a layout to look like?

Grahame Grieve (Sep 18 2020 at 20:48):

all canonical resources should have a summary table as part of the template

Grahame Grieve (Sep 18 2020 at 20:48):

other than that... nothing special

Bob Dolin (Sep 18 2020 at 20:49):

@Lloyd McKenzie I don't really understand why the publishing process adds dataRequirements and parameters to my Library example - presumably that was added to meet someone's needs, and I guess it doesn't really hurt me. What I'm looking to do is [1] Create a profile on FHIR Library resource, that requires a reference to a CQL file - this would just render like any other profile; [2] Create a FHIR Library example, that conforms to the profile, which references a CQL file; [3] Enable the ability of a reviewer of the IG to view the text-based referenced CQL file.

Bryn Rhodes (Sep 18 2020 at 20:51):

@Bob Dolin , where would the reference to the CQL file point?

Bryn Rhodes (Sep 18 2020 at 20:52):

The point of packaging a CQL library as a FHIR Library is that the Library resource actually _contains_ the CQL. Otherwise, to distribute the library, you have to _also_ distribute the CQL file.

Bob Dolin (Sep 18 2020 at 20:55):

@Bryn Rhodes I don't really mind if the CQL is embedded in the FHIR Library, but I also want to maintain the CQL files separately. I was thinking, for the examples, just putting the CQL files into the github input/cql folder, and then putting the filenames into Library.content.url

Bob Dolin (Sep 18 2020 at 20:57):

I may be off on this @Bryn Rhodes , but it seems as though, if you have a lot of CQL files, you can leverage the metadata afforded by the FHIR Library to find the one you want, but there will still be discoverability cases where it'll help to be able to search through the corpus of CQL text files.

Lloyd McKenzie (Sep 18 2020 at 20:57):

The templates don't add anything to the Library. If that's happening, it's happening inside the publisher itself...

Grahame Grieve (Sep 18 2020 at 21:04):

right. I want the template to do something...

Bryn Rhodes (Sep 18 2020 at 21:06):

@Bob Dolin , yes, you still maintain the source as CQL files, but when they are packaged in the implementation guide, they are in FHIR Library resources. The generated narrative contains the CQL so that is searchable/indexable.

Lloyd McKenzie (Sep 18 2020 at 21:07):

@Grahame Grieve I'm referring to Bob's report of additional data elements being inserted into his library.

Lloyd McKenzie (Sep 18 2020 at 21:08):

On the topic of template change - should the summary table appear if the resource is being used as an example, or only as an actual IG conformance resource?

Grahame Grieve (Sep 18 2020 at 21:12):

not sure.

Grahame Grieve (Sep 18 2020 at 21:12):

probably yes

Grahame Grieve (Sep 18 2020 at 21:12):

and setting up the library happens in the publisher loading code

Bob Dolin (Sep 18 2020 at 21:23):

So, let me ask you this - let's say I don't really need the automated packaging of CQL into the FHIR Library. What if I could just reference the CQL files direct in the IG, and then have them show up on the Table of Content and Artifact Summary pages?

Grahame Grieve (Sep 18 2020 at 21:24):

we don't want you to do that, and we think you shouldn't

Bob Dolin (Sep 18 2020 at 21:24):

how come?

Grahame Grieve (Sep 18 2020 at 21:24):

because it avoids all the downstream distribution mechanisms that people use

Bob Dolin (Sep 18 2020 at 21:25):

sorry, not following you. You're saying its harder to maintain the link between the FHIR Library and it's corresponding CQL file if we treat them as separate but referenced files?

Grahame Grieve (Sep 18 2020 at 21:31):

very much so

Grahame Grieve (Sep 18 2020 at 21:31):

I don't see why you want it not to be part of the library?

Bob Dolin (Sep 18 2020 at 21:35):

I don't mind if it's base64 encoded into the FHIR Library really if people find that convenient, but I want to have stand-alone CQL files that I can reference by the FHIR Library. This lets me manage the CQL files, keep them where I want, search them when I can't find the one I want, etc.

Grahame Grieve (Sep 18 2020 at 21:35):

I can't figure out whether you're saying that as an author or a reader

Bob Dolin (Sep 18 2020 at 21:36):

Here's a scenario: I'm a CQL author sitting down to write a new rule, but I want to first see if such a rule already exists.

Bob Dolin (Sep 18 2020 at 21:38):

As for a reader of the IG, I think the current publishing solution works, although if my example FHIR Library includes a content.url, it seems to suppress the nice rendering of the CQL. I'll test it again though...

Grahame Grieve (Sep 18 2020 at 21:39):

yeah the rendering of library needs work.

Grahame Grieve (Sep 18 2020 at 21:39):

But I don't understand

I'm a CQL author sitting down to write a new rule, but I want to first see if such a rule already exists

Bob Dolin (Sep 18 2020 at 21:42):

I was imaging an enterprise CDS management system, containing 100s of rules - for diabetes, for depression, for all kinds of things. When the time comes to write a new rule, say, for all-cause-mortality for congestive heart failure, I'll be darned if I can remember which CQL file contains that rule, since I wrote it about a year ago. It's too granular to be discoverable with FHIR Library metadata, so I just want to do a text-based search of all by CQL files.

Bob Dolin (Sep 18 2020 at 21:51):

When I include this

"content": [
{
"id": "ig-loader-DepressionMeasure.cql"
}

in my FHIR Library, I get all the bells and whistles, including packaging the CQL into the FHIR Library. Seems like this is very important to folks, and it doesn't really hurt anything, so that's fine.

But when I also include contentType and url, like this:

"content": [
{
"id": "ig-loader-DepressionMeasure.cql",
"contentType": "text/cql",
"url": "DepressionMeasure.cql"
}

all the bells and whistles are somehow suppressed. Is it possible to NOT suppress the preferred publishing process while still allowing for the inclusion of contentType and url in my examples?

Bryn Rhodes (Sep 18 2020 at 21:52):

So something like GET [base]/Library?_text=(bone OR liver) and metastases?

Bryn Rhodes (Sep 18 2020 at 21:55):

And actually, I think "all-cause-mortality" and "congestive-heart-failure" are exactly the kinds of concepts you'd want to tag the rule with in the useContext element to make searches exactly like the one you're describing easier.

Bob Dolin (Sep 18 2020 at 21:56):

@Bryn Rhodes I hadn't considered that. You're saying that since the publishing process puts the full text into Library.text, than it's equivalent to full text searching of textual CQL files?

Bryn Rhodes (Sep 18 2020 at 21:57):

It's a superset of that, because the narrative does contain other things, but yes it would search the narrative text.

Bob Dolin (Sep 18 2020 at 21:57):

And as for useContext, yes, I agree that is a valuable element. But look at VSAC for instance - you can search using metadata, and generally hone in on the right value set, but imagine searching value sets via something like 'find value sets that contain code X and do not contain code Y'

Grahame Grieve (Sep 20 2020 at 21:14):

I think that you're making Bryn's point. More generally, you seem to be assuming that indexing etc won't work unless the raw cql source is available, but that seems like a strange assumption to me.

But I spent some time on this, so from the next release:

• Library resources will be rendered much more nicely.
• do not use library.liquid (@Bryn Rhodes - I can do things in the code that you wouldn't try in liquid)
• the contents of the library will be rendered (with syntax highlighting)
• an additional file Library-[id].cql (or whatever) will be produced as well

Grahame Grieve (Sep 20 2020 at 21:15):

so just let the cql file be in the library as we recommend

Bryn Rhodes (Sep 20 2020 at 21:48):

Great, thanks Grahame!

Bob Dolin (Sep 21 2020 at 14:28):

@Grahame Grieve thanks very much. You all have a lot more experience with CQL than I do, so I'll follow your recommendations.

Bob Dolin (Jan 05 2021 at 00:20):

@Bryn Rhodes @Grahame Grieve sorry to keep at this, but I still can't get rendering to work. Here is my repository: https://github.com/HL7/fhir-outcome-criteria-framework-ig; Here is the build log: http://build.fhir.org/ig/HL7/fhir-outcome-criteria-framework-ig/branches/master/build.log. When I build locally (publisher 1.1.56), I get a very nice representation of the CQL that shows up at the bottom of the Library (see picture). But that CQL doesn't show up on the build site, and the build log is giving me error messages I don't understand, such as [Internal error in location for message: 'Error @1, 1: Found / expecting a token name', loc = '/scratch/ig-build-temp-ZH5E1N/repo/input/cql/depressionIntervention.cql', err = 'CQL source was not associated with a library resource in the IG.']. Any advice? image.png

Bryn Rhodes (Jan 06 2021 at 18:49):

Hi @Bob Dolin , apologies for the delay, I'll try to get to this today

Bob Dolin (Jan 06 2021 at 18:55):

Thanks @Bryn Rhodes

Last updated: Apr 12 2022 at 19:14 UTC