FHIR Chat · sushi: Fhirpath escape rules · shorthand

Stream: shorthand

Topic: sushi: Fhirpath escape rules

Patrick Werner (Oct 21 2021 at 15:07):

We are using sushi extensively to create testscript resources containing a lot of FHIRpath Expressions.
Writing fhirpath in FSH is quite cumbersome as you have to pay attention to the escaping rules of FSH.

Patrick Werner (Oct 21 2021 at 15:10):

It would be nice to have a way to just copy in the fhirpath expressions and use them without escaping.

Patrick Werner (Oct 21 2021 at 15:13):

Is there an escaping/un-escaping tool FHIRpath <-> FSH ?

Chris Moesel (Oct 21 2021 at 15:47):

Can you provide some examples of the types of things you're commonly needing to escape?

Yannick Börner (Oct 21 2021 at 15:58):

It is mostly generic FHIRPath expressions that look like this:

component.where(code.coding.system = 'http://loinc.org' and code.coding.code = '8462-4'\).exists(\)
component.select(valueQuantity.code = 'mm[Hg]'\).allTrue(\)
component.all(valueSampledData.exists(\)\)\)
component.all(valueSampledData.origin.exists(\)\)\)
entry.resource.all((code.coding.where(code = '8462-4' and system = 'http://loinc.org'\).exists(\) or component.where(code.coding.where(code = '8462-4' and system = 'http://loinc.org'\)\).exists(\)\) and (valueQuantity.where($this.value = 107\).exists(\) or component.valueQuantity.where($this.value = 107\).exists(\)\)\)

Notice how the escaping gets a bit crazy at times and impacts readability. We also have some cases in which we provide descriptions that contain i.e. a comma which needs to be escaped.

Chris Moesel (Oct 21 2021 at 17:24):

Oh, I see. It looks like you are passing FHIRPath as an argument to parameterized RuleSets? Is that right? Because I don't think you'd need to escape ) and , otherwise.

Patrick Werner (Oct 22 2021 at 07:29):

yes you are right @Chris Moesel

Patrick Werner (Nov 25 2021 at 13:07):

I want to come back to this issue, i ran into this issue again when nesting RuleSets in Rulesets:

RuleSet: inner (linkId)
* item[+].linkId = {linkId}
  * item[+].linkId = {linkId}_question

RuleSet: outer (linkId)
* insert inner ({linkId})

Instance: Test
InstanceOf: Questionnaire
* insert outer ("test\, test1")

Patrick Werner (Nov 25 2021 at 13:07):

The escaping is lost after the first RuleSet -> leading to an error.
Would it be possible for the FSH Syntax/SUSHI to just accept Strings as they are? ( ignore ,and )if they are inside of a "..."block?)

Patrick Werner (Nov 25 2021 at 13:18):

https://fshschool.org/FSHOnline/#/share/3HV93fK

Patrick Werner (Dec 01 2021 at 15:06):

@Chris Moesel can i make this a feature request issue? Or is something from the FSH grammar blocking this idea?

Chris Moesel (Dec 01 2021 at 18:38):

Hi @Patrick Werner -- parameterized rule sets are Trial-Use in FSH, so we do have the liberty to make changes like this. Feel free to propose a change on the FHIR issue tracker (as I think that's where spec feature requests are supposed to go). Make sure you indicate it is for the FHIR Shorthand spec. As for whether or not we do it, I think we'll have to look at the implications and see if it adds new complications somewhere else. I like the idea in principal, but we'll have to see if there are other consequences of making a change like this.

Bas van den Heuvel (Dec 03 2021 at 14:05):

So far as I see, there is nothing in the shorthand specification where it requires parsing of the strings in a general context.
This might just be an implementation feature, which I agree, is very annoying.

Related question: what does this mean for aliases? Will they also be replaced if the alias is in a text string?

Chris Moesel (Dec 03 2021 at 14:38):

So far as I see, there is nothing in the shorthand specification where it requires parsing of the strings in a general context.

I'm not sure exactly what you mean, but in the context of using parameterized RuleSets, the spec only calls out that (a) parameters are wrapped in ( ), (b) each parameter is separated by ,, (c) literal , and ) must be escaped inside a parameter, and (d) whitespace between parameters is removed (doc). The idea was to try to simplify the rules such that it really works like raw find/replace would. Once we get into anything much more sophisticated, we need to treat parameters more like tokens, which complicates matters in some ways and also may restrict where placeholders can be in the RuleSet.

For example, imagine a RuleSet that contains a string like this: "This is a profile for {Description}". As it is defined today, you would just pass in a patient -- which would get substituted via simple find/replace. But if we support special tokenization of the parameters (to support things like allowing strings with unescaped , in them), then you'd pass in "a patient" -- but it's no longer find/replace because we now need to be smart enough to know if we keep the wrapping " (when it represents a whole string in the RuleSet) or we discard the wrapping " (when it is inside a string in the RuleSet). This requires a more complicated parsing of the RuleSet before substitution. That is, unless we also make a new rule that " is always stripped before substitution -- but that can also be confusing to users. So... it's not as clear as it might seem.

what does this mean for aliases? Will they also be replaced if the alias is in a text string?

Aliases are only allowed where a URL token would be expected in the grammar, so an alias reference inside a string would not be replaced (e.g. "The URI for LOINC is $LOINC." would not replace the $LOINC alias with the LOINC URI).

Bas van den Heuvel (Dec 03 2021 at 15:16):

Chris,
Thanks for the response. If I understand your correctly, a rule such as stated below should just work.

• insert fhirPathVariable( "score", "%resource.repeat(item).where(linkId = 'score').answer.value")

Yet, when I run this through sushi I get the following errors:
Importing FSH text...

error extraneous input '.where(linkId' expecting {<EOF>, KW_ALIAS, KW_PROFILE, KW_EXTENSION, KW_INSTANCE, KW_INVARIANT, KW_VALUESET, KW_CODESYSTEM, KW_RULESET, KW_MAPPING, KW_LOGICAL, KW_RESOURCE}

error Cannot find definition for Instance: "%resource.repeat(item. Skipping rule.

Chris Moesel (Dec 03 2021 at 16:06):

No... I think that is what was being proposed as something to consider. As it is today, however, you need to escape all ) except the last one so they don't get interpreted as the end of the argument list. So you need something more like: insert fhirPathVariable( "score", "%resource.repeat(item\).where(linkId = 'score'\).answer.value") (assuming the wrapping " are also intended to be literally inserted into the RuleSet).

Bas van den Heuvel (Dec 06 2021 at 08:23):

No... I think that is what was being proposed as something to consider. As it is today, however, you need to escape all ) except the last one so they don't get interpreted as the end of the argument list. So you need something more like: insert fhirPathVariable( "score", "%resource.repeat(item\).where(linkId = 'score'\).answer.value") (assuming the wrapping " are also intended to be literally inserted into the RuleSet)

Chris, I think I understand the issue (and got mine to work). Thanks.
As you indicated above the escapes are needed in order for the parser to identify the different input parameters. Too bad this causes a somewhat strangly formatted input format. The root cause behind this that for parameterised Rule inserts we seem to make a format shift. From the shorthand - one parameter on a line starting with an '*' to a more java/C-style format (I know that it is present in Reference(xx) like format but these are behind an '=').
This shift is responsible that input intended for the 'shorthand' domain is to interpreted in the 'java-like' domain. The escapes are needed to resolve this.

This make me wonder what the more shorthand way of addressing this would be.
A ruleset specified as

RuleSet: variable( name, language, expression ) * extension[+]
* url = "http://hl7.org/fhir/StructureDefinition/variable"
* valueExpression
* name = {name}
* language = {language}
* expression = {expression}

is now inserted as:

• insert variable( "varNam", #text/fhirpath, "%resource.repeat(item\).answer.value.extension.value.aggregate($this+$total\,0\)" )

Note the escaped characters. As these are removed when the rule is nested, this makes nesting of such expressions difficult.
From a shorthand perspective, the trailing ')' has no direct meaning as the end of the statement is determined by the next line starting with an '*'.

A more shorthand approach could be:

• insert variable
  • name = "varNam"
  • language = #text/fhirpath
  • expression = "%resource.repeat(item).answer.value.extension.value.aggregate($this+$total,0)"

This would be more verbose but would retain the shorthand way of providing data. It would also allow nesting of rules as the beginning and end of each parameter is obvious. Wouldn't this be a better way to provide the parameters? Or an alternative if the bracketted approach does not work?

Chris Moesel (Dec 06 2021 at 14:27):

Thanks for the suggestion, @Bas van den Heuvel. That is an interesting approach -- and one that we had not yet considered. There are a few implications to consider:

• * would need to be escaped in parameter values (unless we restricted the values to only allow valid tokens like a string, number, code, etc.)
• It requires indenting -- which so far we've kept as a purely optional feature since some people really don't like it. (E.g., in other cases, there is always a way to do something without indenting, but in this case, you must use indenting).

I'm making a note to revisit parameterized RuleSets -- at which point we'll look at your suggestion, as well as Patrick's and anyone else who has ideas on how to improve this!

Elliot Silver (Dec 06 2021 at 16:39):

Reading this thread, I wonder if two notations are needed? One would substitute the parameter as is; the other strips off a level of quoting and escaping. (Alternatively, one notation substitutes the parameter as is; the other adds quotes and escapes.) Something along the lines the following, using ! to indicate the new behaviour:

RuleSet: variable( name, language, expression )

• extension[+]
  • ...
  • name = {name}
  • language = {language}
  • expression = {!expression}

or

insert variable( "varNam", #text/fhirpath, !"%resource.repeat(item).answer.value.extension.value.aggregate($this+$total,0)" )

The second version might actually be generally useful to outside of this context--take a string and escape it properly. I haven't had my coffee yet, or thought it through fully, but is there some value here?

Last updated: Apr 12 2022 at 19:14 UTC