FHIR Chat · Proposal: Indent Blocks · shorthand

Stream: shorthand

Topic: Proposal: Indent Blocks

Chris Moesel (Mar 11 2021 at 02:41):

We've previously discussed the idea of supporting path-based context for rules via keywords such as with and modified rule markers such as **, ***, etc. But... we never quite felt we had landed on the right solution. Over the past few weeks, we've continued to iterate over different syntaxes trying to find the best solution -- and now have something to propose. We think that those of you who enjoy YAML (ahem, @Eric Haas) will probably like this proposal. We're interested in what the rest of you think (and you too, Eric).

In short, the proposal is to indicate path context and structure via indented blocks of rules. As an example, these FSH STU1 rules:

* name 1..*
* name.given 1..*
* name.family 1..1
* active 1..1

could be written using "indent blocks" like this:

* name 1..*
  * given 1..*
  * family 1..1
* active 1..1

This means whitespace before * becomes significant, but we like it because it reminds us of markdown lists (with nested items) and, as previously noted, YAML.

Chris Moesel (Mar 11 2021 at 02:44):

We also propose using this syntax to expand the CodeSystem grammar to allow access to other concept properties (e.g., designation) and to author hierarchical code systems. Using a very abbreviated version of US Core's Race & Ethnicity CDC code system, for example:

CodeSystem: RaceAndEthnicityCDC
* ^property.code = #abstract
* ^property.description = "True if an element is considered 'abstract'"
* ^property.type = #boolean
* #1000-9 "Race" "Race, Note that this is an abstract 'grouping' concept and not for use as a real concept"
  * ^property.code = #abstract
  * ^property.valueBoolean = true
  * #1002-5 "American Indian or Alaska Native" "American Indian or Alaska Native"
    * #1004-1 "American Indian" "American Indian"
    * #1735-0 "Alaska Native" "Alaska Native"
  * #2028-9 "Asian" "Asian"
    * #2029-7 "Asian Indian" "Asian Indian"
    * #2030-5 "Bangladeshi" "Bandgladeshi"
* #2133-7 "Ethnicity" "Ethnicity Note that this is an abstract 'grouping' concept and not for use as a real concept"
  * ^property.code = #abstract
  * ^property.valueBoolean = true
  * #2135-2 "Hispanic or Latino" "Hispanic or Latino"
    * #2137-8 "Spaniard" "Spaniard"
    * #2148-5 "Mexican" "Mexican"
  * #2186-5 "Not Hispanic or Latino" """
      Note that this term remains in the table for completeness, even though within HL7, the notion of
      "not otherwise coded" term is deprecated.
      """

Chris Moesel (Mar 11 2021 at 02:45):

For a more complete description and further examples, see our Proposed STU2 Feature: Indent Blocks wiki page.

David Pyke (Mar 11 2021 at 14:25):

Can we change to 4spaces/1 tab (despite Eric's love for YAML)? That's a convention far beyond YAML and makes life easier for users.

David Pyke (Mar 11 2021 at 14:26):

In general, like this proposal. It makes things easier on everyone.

Chris Moesel (Mar 11 2021 at 14:33):

YAML actually is not prescriptive about the number of spaces (it's up to the author and YAML interpreters just adjust to what they see) -- so in that way, variable spaces would be more like YAML. OTOH, YAML specifically forbids tabs.

I think 2 spaces vs 4 spaces is definitely a personal preference. I exclusively use 2 spaces for all markup style languages (and many non-markup languages) -- and I know I'm not alone. So I'd personally lean more toward making it flexible (so authors can use their preference) rather than changing the requirement from 2 to 4 spaces outright. Thanks for the feedback!

John Moehrke (Mar 11 2021 at 14:40):

+1 on 2

John Silva (Mar 11 2021 at 16:05):

I'm not a fan of languages like YAML that depend on spaces to infer meaning. In many editors it's hard to see spaces (vs tabs) and it seems somewhat 'obscure' to depend on spacing/indentation to imply meaning. Is there something that could be used that is more obvious?

Jean Duteau (Mar 11 2021 at 16:06):

obviously not a Python programmer! :)

John Silva (Mar 11 2021 at 16:08):

Yeh, did one of those online "Learn Python" classes and felt the spacing stuff made learning the language more painful that it should be! ;-). (Yes, a good language-sensitive editor can help but WHY depend on white space that's 'invisible'?)

Eric Haas (Mar 11 2021 at 16:11):

I think it makes sense and it more Markdown like than YAML like and who doesn't embrace markdown. It makes is a bit more readable too IMO

Jose Costa Teixeira (Mar 11 2021 at 17:35):

indentation or ** are fine for me. I'll get used to either (would prefer **)

Jose Costa Teixeira (Mar 11 2021 at 17:36):

@Chris Moesel in your analysis you mention ambiguous or nonsensical indexing - did you find a case for ambiguity?

Jose Costa Teixeira (Mar 11 2021 at 17:36):

(that was the reason for thewith - to avoid ambiguity.)

Jose Costa Teixeira (Mar 11 2021 at 18:19):

Here's an example: If I have a questionnaire,

* item[+].id = "question1"
* item[+].id = "question2"
* item[+].item[+].id = "question2-1"
* item[+].item[+].item[+].id = "question2-1-1"
* item[+].item[+].item[+].item[+].id = "question2-1-1-1"
* item[+].item[+].id = "question2-2"

How would this look like?

* item[+]
  *id = "question1"
  * item[+]
    * id = "question2"
    * item[+]
      * id = "question2-1"
      * item[+]
        * id = "question2-1-1"
        * item[+]
          * id = "question2-1-1-1"
  * item[+]
    * id = "question2-2"

Elliot Silver (Mar 11 2021 at 18:35):

I like the proposal and find it interesting (or telling, depending on your point of view) that most of the discussion so far is about whitespace, rather than the meat of the change. I also like the related insert rules change.

Elliot Silver (Mar 11 2021 at 18:37):

The one thing I note is that you introduce the situation of having an element name without saying anything further about it. Is there a requirement that if you have

* item

that there must be an indent block after it?

Chris Moesel (Mar 11 2021 at 18:38):

@Jose Costa Teixeira -- Since question 1 and question 2 are siblings, they should be at the same indent level (i.e., question 2 is not a child of question 1). So if I were to write your original FSH using the new indent block syntax, I would write it as:

Chris Moesel (Mar 11 2021 at 18:43):

Wait, @Jose Costa Teixeira . Actually, I'm not sure your original FSH is really what you want, I think you have a few too many [+] in there. I think you want some of those to be[=]. So, if the original FSH is:

* item[+].id = "question1"
* item[+].id = "question2"
* item[=].item[+].id = "question2-1"
* item[=].item[=].item[+].id = "question2-1-1"
* item[=].item[=].item[=].item[+].id = "question2-1-1-1"
* item[=].item[+].id = "question2-2"

then the indented version is:

* item[+].id = "question1"
* item[+]
  * id = "question2"
  * item[+]
    * id = "question2-1"
    * item[+]
      * id = "question2-1-1"
      * item[+].id = "question2-1-1-1"
  * item[+].id = "question2-2"

or if you choose to just use indenting even where there is only one child (which is technically not necessary but maybe you like the consistency) then:

* item[+]
  * id = "question1"
* item[+]
  * id = "question2"
  * item[+]
    * id = "question2-1"
    * item[+]
      * id = "question2-1-1"
      * item[+]
        * id = "question2-1-1-1"
  * item[+]
    * id = "question2-2"

Chris Moesel (Mar 11 2021 at 18:45):

Elliot Silver said:

The one thing I note is that you introduce the situation of having an element name without saying anything further about it. Is there a requirement that if you have

* item

that there must be an indent block after it?

Without a following indent block, that rule would essentially be a no-op. I'm not sure if we would disallow it, but it's probably worthy of a warning since it adds no value.

Elliot Silver (Mar 11 2021 at 18:54):

Makes sense. What about:

* item[+]

Chris Moesel (Mar 11 2021 at 18:59):

Ha. Yeah, I guess that is not a no-op, and perhaps some authors might prefer to use a rule only to increment an index (and then use item[=] in rules after). I hadn't thought of that, but it seems valid.

Chris Moesel (Mar 11 2021 at 19:06):

John Silva said:

I'm not a fan of languages like YAML that depend on spaces to infer meaning. In many editors it's hard to see spaces (vs tabs) and it seems somewhat 'obscure' to depend on spacing/indentation to imply meaning. Is there something that could be used that is more obvious?

Yeah, I get that -- meaningful whitespace can be a controversial topic. I'm also not a fan in many contexts, but in some cases it feels more natural than others. This seems like a tangent, but... In last week's HL7 CDS Workgroup meeting, someone shared a Word document with a draft of the MetadataResource in it. We were already narrowing in on proposing indents for structure -- but looking at this document confirmed that it is a natural way that people think about and sketch out these things. In fact, I was very surprised to realize that the structure looked almost exactly like it would in FSH if we combined the logical model and indent block proposals. I just had to take a screenshot:
image.png

Chris Moesel (Mar 11 2021 at 19:08):

Ultimately we want FSH to feel natural. We tried lots of things (multiple *, various placeholders, path aliases, etc) -- and in the end, this is what felt most natural to us. But this is why we bring it to the community -- to see if it's just us or if others feel the same way.

Jose Costa Teixeira (Mar 11 2021 at 19:11):

@Chris Moesel You are right, I had too many [+], some of them were [=].

Chris Moesel (Mar 11 2021 at 19:11):

From a practical standpoint, many editors can be configured to show whitespace characters (in a fairly unobtrusive way). But I'd also note that what we are proposing is an optional syntax. Authors can still choose to use the long-form syntax. (With one exception -- we didn't propose a long-form syntax for the new CodeSystem features yet, but I was actually just tossing around some ideas for that this morning).

Jose Costa Teixeira (Mar 11 2021 at 19:11):

About the no-op, should we do the yaml thing and add a :?

Jose Costa Teixeira (Mar 11 2021 at 19:11):

or it's the indentation that will work?

Chris Moesel (Mar 11 2021 at 19:12):

The indentation is enough for us to correctly process it, but if we felt we wanted to require a : to make the author's intent clear ("I meant to do this") then I think that's a possibility.

Jose Costa Teixeira (Mar 11 2021 at 19:12):

danger with indentation is that IDEs sometimes think they know the indentation we want. And Python suffers a bit with that.

Jose Costa Teixeira (Mar 11 2021 at 19:13):

Ok the : is more to give a sense of what we are going to do next, so that when copy-pasting lines the whole thing still makes sense.

Jose Costa Teixeira (Mar 11 2021 at 19:13):

i mean it as a suggestion

Chris Moesel (Mar 11 2021 at 19:14):

Re: IDEs. Yeah, that's a blessing and a curse. Editor-assist for indentation can be really useful! Sometimes it's nice to hit enter and have it already be indented for me. But yeah, that also means the editor is guessing what you want -- and sometimes you really want to outdent.

Chris Moesel (Mar 11 2021 at 19:23):

The good (?) thing about indents in this context (vs python) is that most incorrect indents would be caught by SUSHI because they often result in invalid paths. My issue w/ space-sensitive programming languages is that I always ended up messing up the scope of my for-loops, and those errors were never apparent until runtime (if at all). Luckily FSH wouldn't suffer from that as much (although item is arguably one place where you could run into that).

Chris Moesel (Mar 16 2021 at 20:48):

Just bumping this up one more time to see if there is any more feedback. If not, this is a Standard for Trial Use after all, so perhaps we will just try it to see how it works out (possibly adjusting for the feedback we've received so far). As with any HL7 standard, it will need to go through ballot before it is official anyway. But still, we like to get things right the first time since we have active users who will integrate these features into their IGs before ballot. So... anyone else have thoughts?

Jose Costa Teixeira (Mar 16 2021 at 20:56):

"So... anyone else have thoughts?" or "So... anyone have more thoughts?" ? :grinning:

Jose Costa Teixeira (Mar 16 2021 at 20:57):

I'd make sure there's no chance for ambiguity to be a problem in the future (on nested elements in questionnaires, plandefinitions, etc)

Chris Moesel (Mar 16 2021 at 21:08):

Ha. @Jose Costa Teixeira -- I'm interested in further thoughts if you have them too -- but so far only 7 people have weighed in at all and I'd love to get some additional data points and perspectives.

Jose Costa Teixeira (Mar 16 2021 at 21:15):

:) For me it's fine, and I think it's good to have insight from more people.

ryan moehrke (Mar 16 2021 at 21:53):

personally I'm not sure if I'll actually use it because I'm not a huge fan of indentation levels as code rather than ease-of-eyes formatting, but I see the draw so I have no reason to stop this from happening. Regardless of that I had a couple questions I've been stewing on

if how much whitespace means one indent is dependent on how the author uses said whitespace, does this run into any issues with rulesets injecting in code? can one ruleset be used in different indentation levels? how would that look if yes? if no could you pass in the amount of whitespace to use as a base in a parameter? could that be inherited based on how much whitespace is before an "insert" rule?
how do multi-line strings work with these indentations? does the opening stay indented and the rest just sit weirdly left-aligned? (now looking at the spec thus-far you should be able to indent as much as you want and just have the shared indent removed during processing so it shouldn't be an issue)
And also I feel like this got answered but I can't find it anymore, is space the only whitespace character allowed for these indents? or would tab work too? (and if so what happens if you have mixed space+tab indents? an error? an assumed conversion rate?)

Jose Costa Teixeira (Mar 16 2021 at 22:05):

Good point @ryan moehrke
I think the impact on rules and multiline strings should be clarified.
For tabs / spaces, I would follow the yaml rule - no tabs

Elliot Silver (Mar 17 2021 at 03:46):

I think the behavior presented for rules was that the rule invocation inherits the indent level of where it is inserted.

I would hypothesize that It isn't possible for rule contents to be less indented than where it was invoked. And, that It is possible to indent further within a rule, but it is never (irrespective of use of rule) permissible to skip indent levels.

I have a certain reluctance around whitespace-sensitive languages too, but I recognize that FSH is already in that category, so am willing to embrace it.

Mark Kramer (Jun 05 2021 at 15:49):

Just to bring this thread to a satisfying close, this feature has been added to SUSHI 2.0 and is now available, with description located here. Thank you all for the very vigorous and helpful discussion.

Jose Costa Teixeira (Jun 13 2021 at 14:55):

I can't make indentation work with rules.

Jose Costa Teixeira (Jun 13 2021 at 14:56):

has this been considered as part of the previous discussion?

Jose Costa Teixeira (Jun 13 2021 at 14:56):

here's one example which tries too hard to get indentation and rule working:
https://fshschool.org/FSHOnline/#/share/35hnIQk

Jose Costa Teixeira (Jun 13 2021 at 14:57):

I may be missing something. If not, I think this language feature should include these cases. As a language, FSH should be consistent, right?

Jose Costa Teixeira (Jun 13 2021 at 14:58):

Perhaps it is just the parser that does not do anything yet, but what are the FHIR Shorthand rules around this?

Nick Freiter (Jun 14 2021 at 12:51):

Indentation is not supposed to work in that context. From the spec:

The full path of all rules is resolved from the context specified by indentation before any rules are applied.

This means that indentation is resolved before an insert rule is applied.

Chris Moesel (Jun 14 2021 at 13:10):

We felt it would be too confusing to have RuleSets with rules that start indented -- because they're so far from the context in which they are used. That said, we have discussed the possibility of allowing the insert rule itself to be indented to indicate that's the context in which the RuleSet rules should be applied. I see you tried to do that at the end of your example; so that's something that we might (probably will) address.

Nick Freiter (Jun 14 2021 at 13:35):

Something to consider though is that in the example @Jose Costa Teixeira linked it looks like he wanted to use the rules from one insert rule to set the context in the second insert rule. I'm not sure exactly how that fits into the possibility @Chris Moesel mentioned. And I'm interested to hear what you think about that case Chris.

Chris Moesel (Jun 14 2021 at 15:17):

@Nick Freiter - Ah, I don't think I realized that was what he was going for. I think that RuleSets should not "leak" context; i.e., any context in the RuleSet ends at the end of the RuleSet. That way when you're looking at a set of rules with insert rules, it's still easy to see the overall structure without needing to know the details of the RuleSet being applied. I think it would be confusing otherwise. So... I think the part of the spec you quoted above still applies.

Chris Moesel (Jun 14 2021 at 15:19):

But maybe we need to call that out even more explicitly in the spec so it is crystal clear. Since we sometimes refer to insert like copy-pasting the RuleSet right in place, I can see why there might be confusion.

Jose Costa Teixeira (Jun 14 2021 at 20:13):

this may be a better example:
https://fshschool.org/FSHOnline/#/share/3zowzxv

Jose Costa Teixeira (Jun 14 2021 at 20:13):

I should not have indented things inside the ruleset

Jose Costa Teixeira (Jun 14 2021 at 20:14):

but my purpose is to add an insert inside another insert

Jose Costa Teixeira (Jun 14 2021 at 20:14):

so indent the calling of rulesets

Chris Moesel (Jun 14 2021 at 20:32):

Yes, @Jose Costa Teixeira, if I understand your example correctly, this is something we hope to support as part of STU2; we just have not got there yet. But I think in real use, we would simplify your example even further to something like this, right? https://fshschool.org/FSHOnline/#/share/3xkbaDR

Jose Costa Teixeira (Jun 14 2021 at 21:02):

yes, that's it. In my original idea I had 2 attempts at doing that. The simpler one is the one that would work best.

Jose Costa Teixeira (Jun 14 2021 at 21:02):

Is that compatible with the syntax defined?

Chris Moesel (Jun 14 2021 at 21:35):

Right now, the spec on build.fhir.org says that you cannot indent insert rules, because thus far, insert rules have not been allowed to be applied to specific paths (e.g., right now insert rules are always "path-less"). The indent syntax would allow for what I showed in the example linked above, but first we need to introduce a form of insert that allows for insertion at a path, and then we can extend the specification of the indent syntax to leverage that.

Chris Moesel (Jun 14 2021 at 21:37):

No. Wait. I take that back. Sorry, I was a bit distracted and hadn't fully thought through that. My example above is not quite right because it still leverages paths introduced by the insert rule above it. We don't (and likely won't) allow that. Let me draft a better example for you.

Chris Moesel (Jun 14 2021 at 21:42):

Since item is actually the thing that is variable (could be any level deep), it's best to actually not include that in the RuleSet definition, but rather specify it as part of the path to which the RuleSet is applied. E.g.: https://fshschool.org/FSHOnline/#/share/35jTe03

Jose Costa Teixeira (Jun 14 2021 at 22:28):

looks almost like a "with" statement but on the other end :)

Jose Costa Teixeira (Jun 14 2021 at 22:30):

so instead of putting the context in the beginning as I was thinking for the with, you pass the context when calling the rule?

Jose Costa Teixeira (Jun 14 2021 at 22:30):

i.e. the logic for this is:

Jose Costa Teixeira (Jun 14 2021 at 22:33):

the insert rule takes its context from where it is called, if there is an explicit declaration of context.
* insert Rule(a,b,c) - there's no context, so this is top level
* context insert Rule(a,b,c) there is a context here, so the parser will determine that context and then apply the rule
* context insert Rule(a,b,c) there is a context and indentation here, so the parser will determine where that context applies and then apply the rule

Jose Costa Teixeira (Jun 14 2021 at 22:33):

is that what you mean?

Jose Costa Teixeira (Jun 14 2021 at 22:43):

I think there may still a problem with this: won't the insert of a ruleset create a context itself? What happens?
https://fshschool.org/FSHOnline/#/share/2TptEEh

Nick Freiter (Jun 15 2021 at 13:23):

If we do add support for paths in an insert rule, the context for subsequent rules indented below an insert rule will take their context from the explicit path on that insert rule, not from the implicit paths being inserted. So in this example:

* item[0] insert Question(patient,Patient,group,false)
  * item[0] insert Question(status,IndentedStatus,group,false)

It is equivalent to saying:

* item[0] insert Question(patient,Patient,group,false)
* item[0].item[0] insert Question(status,IndentedStatus,group,false)

The paths are resolved before applying the insert rule, so the insert of the RuleSet does not create any context.

Jose Costa Teixeira (Jun 15 2021 at 20:05):

I think that rule makes sense

Jose Costa Teixeira (Jun 15 2021 at 20:06):

as for the syntax, wouldn't it be better to delimit the context?

Jose Costa Teixeira (Jun 15 2021 at 20:07):

in the example, item[0] is not a keyword, so perhaps isolate that? as in

* item[0]: insert Question(patient,Patient,group,false)
  * item[0]: insert Question(status,IndentedStatus,group,false)

Jose Costa Teixeira (Jun 15 2021 at 20:08):

that would also help in those rare cases where the attribute has the same name as a fsh keyword

Chris Moesel (Jun 16 2021 at 22:38):

Thanks, @Jose Costa Teixeira. TBH, I'm not sure if the trailing : adds much. Other rules w/ infix keywords don't use it (e.g., identifier obey inv-2, component contains systolic 1..1). I also don't think we need the : to disambiguate anything in the grammar (even in the rare case of a path named insert). So... I feel like it would be easier for users to remember if it just did the same as other rules: path keyword value(s). E.g.:

* item[0] insert Question(patient,Patient,group,false)
  * item[0] insert Question(status,IndentedStatus,group,false)

Chris Moesel (Jun 16 2021 at 22:42):

For anyone who wants to track this, this capability is (more or less) covered by SUSHI#828.

Jose Costa Teixeira (Aug 18 2021 at 09:29):

How do I use indentation here?
https://fshschool.org/FSHOnline/#/share/3k3gjeD

Jose Costa Teixeira (Aug 18 2021 at 09:36):

I am trying to indent a process.step inside another process.step, not inside the process.step.operation

Nick Freiter (Aug 18 2021 at 13:15):

Here is an example that matches yours, but used indentation to nest a process.step inside process.step. A similar pattern could be followed with the rest of the file. The indented part starts on line 65

Jose Costa Teixeira (Aug 18 2021 at 19:38):

great. I thought that every line would need to assign a value to a property. if not, that solves the issue.
Thanks!

Last updated: Apr 12 2022 at 19:14 UTC