FHIR Chat · Announcing SUSHI 1.0.0 Beta 1 · shorthand

Stream: shorthand

Topic: Announcing SUSHI 1.0.0 Beta 1

Chris Moesel (Oct 02 2020 at 19:35):

We've been waiting a while for this and are super excited to announce the availability of SUSHI 1.0.0 Beta 1!

SUSHI 1.0.0 Beta 1 is the first beta release of SUSHI 1.0, with full support for FHIR Shorthand 1.0 and a new project structure to better support IG Publisher integration. We could really use some beta testers and would love your feedback. We're particularly interested in:

1. Did this break your existing project? (i.e., does your project compile different with SUSHI 1.0.0 Beta 1 than with SUSHI 0.16.0)
2. Were you able to successfully migrate to the new project structure?

There's a lot that has changed regarding project structure, as well as some deprecated features, and there is a different command to install the beta (since it is not an official release), so... please checkout the Release Notes and SUSHI 1.0.0 Beta Documentation.

David Hay (Oct 03 2020 at 00:11):

When do you expect to fix the issue with the auto-builder? If I read the release notes correctly, then we can't publish to the build environment via git commit... (ie the git webhook...)

Jose Costa Teixeira (Oct 03 2020 at 08:23):

(deleted)

Jose Costa Teixeira (Oct 03 2020 at 08:26):

(deleted)

Jose Costa Teixeira (Oct 03 2020 at 08:28):

(deleted)

Jose Costa Teixeira (Oct 03 2020 at 08:28):

(deleted)

Jose Costa Teixeira (Oct 03 2020 at 08:33):

(deleted)

Chris Moesel (Oct 04 2020 at 16:36):

@David Hay -- You're right -- until it's supported in Auto-Build, it won't work correctly w/ the Auto-Build process that's invoked via the webhook.
We're working out the details of how it should work, but there are some questions around what this means for publishing official versions, etc. If you want to follow along, here's the thread: https://chat.fhir.org/#narrow/stream/179252-IG-creation/topic/Selecting.20SUSHI.20Versions.20in.20AutoBuild

Eric Haas (Oct 05 2020 at 22:31):

OK I am using the beta version and converted my test IG over and updated my custom scripts. Is a simple matter of adding an ig parameter to point to the fsh-generated folder for resources and pages to add the fsh content to my custom content in the input directory. Also a simple bash to modify adding ig.yml to the input/data and running using the autobuild. ( without running sushi ... never thought that was a great feature anyway...)

Eric Haas (Oct 05 2020 at 22:34):

I like the new simpler structure, everything working ok .. so far... I miss using a single YAML file for the ig, ini, and package -list.json but oh well.

Shovan Roy (Oct 06 2020 at 00:33):

@Chris Moesel I took the beta version this morning and flowing the latest folder structure as per the guideline. Now I can see that all the artifacts are being generated under the folder 'fsh-generated\resources'. where as if I'm not wrong then in earlier versions, it used to generate under specific folders: 'examples, profiles, vocabulary'-- is this by design? is there a way (any configuration) to get those artifacts specifically( profiles, examples and vocabulary) generated in specific folders?

Chris Moesel (Oct 06 2020 at 12:27):

Hi @Shovan Roy -- thanks for the feedback. Currently there is not a way to make SUSHI distribute the files into separate folders. We decided to put them all into resources for simplicity, but if you and others prefer them being split, that is good information to know. This is something we could revisit -- but it will also require additional changes to the base template. If you don't mind me asking, why do you like them split? Is it just to make it easier for you to personally review them?

Jose Costa Teixeira (Oct 06 2020 at 13:20):

I'd be happy if sushi put the files to the folders defined in the base template.

• profiles (structuredefs for profiles)
• extensions (structuredefs for extensions)
• vocabulary (NamingSystem, CodeSystem, ValueSet)
• models (structuredefs for LogicalModels)
• capabilities
• operations
• maps (concept and structure)
• testing (testscript, testreport)
• history (for history provenance/bundles)

Jose Costa Teixeira (Oct 06 2020 at 13:24):

I'd be even happier if sushi introduces a feature that allows me to map this and filenames, e.g.
i could chose if my generated extensions go into resources/StructureDefinition-xxxx(as is now) or into extensions/extension-xxx (in case I don't need the StructureDefinition prefix)

Eric Haas (Oct 06 2020 at 14:22):

Im with Chris on this one, I like them in one folder ...

Lets not overcomplicate this this because:

1. in most cases your source is the .fsh file and not the generated-output (Hell it is even recommended to add the directory to the .gitignore file) you can use as many or as few .fsh files as you want and group how you like therein. (e.g. my_awesome_proifles.fhs, my_fabulous_examples.fsh)
2. The IG publisher does not care, it defaults to the /resources folder and you are stuck pointing to all the folders anyway in the the implementation guide folder parameters.
3. Not everyone is using the templates + ig-publisher so lets not have it dictate this behaviour. - They should be separate concerns! Otherwise why remove the integration of the ini and package-list from the the config.yaml file?
4. It make lifes more difficult for those of us who use our own scripts to process the output.

Jose Costa Teixeira (Oct 06 2020 at 14:40):

It's ok to like them in one folder, but it's also ok to like things separate, right?

Jose Costa Teixeira (Oct 06 2020 at 14:40):

I keep them separate for reasons of work segmentation and ownership - the team that makes logical models is not the same that makes testing artifacts

Jose Costa Teixeira (Oct 06 2020 at 14:42):

1. the fsh file is not the single source. I have logical models outside fsh, I have to manage codesystems outside fsh..I don't see the issue

Jose Costa Teixeira (Oct 06 2020 at 14:42):

2. The base template uses the folders I mentioned above

Jose Costa Teixeira (Oct 06 2020 at 14:43):

3. I don't understand- you don't use the base templates?

Jose Costa Teixeira (Oct 06 2020 at 14:45):

4. Also don't understand - the resources folder is supported, as well as the separated folders. Why would we abandon the separate folders? If you want to use one folder, you can

Jose Costa Teixeira (Oct 06 2020 at 14:46):

About the file names - the publisher supports different file name patterns, my suggestion is for sushi to do the same.

Jose Costa Teixeira (Oct 06 2020 at 14:47):

Those patterns are hard coded and I think that is ok

Chris Moesel (Oct 06 2020 at 15:05):

About the file names - the publisher supports different file name patterns, my suggestion is for sushi to do the same.

Last time I tried using different file name patterns, it did not work at all (and that wasn't very long ago). There were a few patterns you could select from, but they were far from arbitrary.

Jose Costa Teixeira (Oct 06 2020 at 15:47):

1. using resourcesor using the other folders (profiles, etc) - that is what we have now, I'd like sushi to be able to split the files. If not, no worries, I can move them by hand and I expect that sushi will avoid creating duplicates. So I don't see an issue.

Jose Costa Teixeira (Oct 06 2020 at 15:51):

2. the filenames pattern - would be a more advanced, later feature that should be discussed before adding. If I have a CodeSystem whose id is "mycodes", I can save it as resources/CodeSystem-mycodes.json or vocabulary/CodeSystem-mycodes.json and IIRC, also as vocabulary/CodeSystem/mycodes.json. I like that.

Eric Haas (Oct 06 2020 at 17:11):

here is my file tree, I manage both my files and fsh files. the fsh-generated files are separate from the ones I manage as they should be. we are not talking about how you manage your file and they are not merged with the fsh-generated ones.

and shockingly yes ...you don't have to publish an IG using the templates or ig-pub for that matter so sushi is not married to it.

├── FHIR-us-davinci-cdex.xml
├── README.md
├── SUSHI-GENERATED-FILES.md
├── _gencontinuous.bat
├── _gencontinuous.sh
├── _genonce.bat
├── _genonce.sh
├── _updatePublisher.bat
├── _updatePublisher.sh
├── fsh-generated
│   ├── includes
│   └── resources
├── ig.ini
├── input
│   ├── data
│   ├── draft
│   ├── examples
│   ├── fsh
│   ├── images
│   ├── includes
│   ├── pagecontent
│   ├── resources
│   └── yaml-resources

Jose Costa Teixeira (Oct 06 2020 at 18:56):

ok. is there anything that doesn't work?

Jose Costa Teixeira (Oct 06 2020 at 18:57):

If the question is about segmenting resources in folders, it is not mandatory, but it is very much appreciated - my point on that one is that whether sushi wants to split the resources or not, it's fine for the template.

Jose Costa Teixeira (Oct 06 2020 at 18:58):

If sushi wants to support yet another structure of resources, that would be messy, and at that moment, I'd say it's probably good to consider a file naming pattern.

Eric Haas (Oct 06 2020 at 19:13):

Sushi is prepending text into markdown files in my input/includes directory each time I run it.

<!-- example-list-generator.md {% comment %}
*****************************************************************************************
*                            WARNING: DO NOT EDIT THIS FILE                             *
*                                                                                       *
* This file is generated by SUSHI. Any edits you make to this file will be overwritten. *
*                                                                                       *
* To change the contents of this file, edit the original source file at:                *
* Davinci-CDEX/input/includes/example-list-generator.md                                 *
*****************************************************************************************
{% endcomment %} -->
<!-- example-list-generator.md {% comment %}
*****************************************************************************************
*                            WARNING: DO NOT EDIT THIS FILE                             *
*                                                                                       *
* This file is generated by SUSHI. Any edits you make to this file will be overwritten. *
*                                                                                       *
* To change the contents of this file, edit the original source file at:                *
* Davinci-CDEX/input/includes/example-list-generator.md                                 *
*****************************************************************************************
{% endcomment %} -->
<!-- example-list-generator.md {% comment %}
*****************************************************************************************
*                            WARNING: DO NOT EDIT THIS FILE                             *
*                                                                                       *
* This file is generated by SUSHI. Any edits you make to this file will be overwritten. *
*                                                                                       *
* To change the contents of this file, edit the original source file at:                *
* Davinci-CDEX/input/includes/example-list-generator.md                                 *
*****************************************************************************************
{% endcomment %} -->

Chris Moesel (Oct 06 2020 at 19:33):

@Eric Haas -- yikes. That doesn't look good. We don't intend to modify anything in input/* -- but it's possible that this is somehow sneaking in because we're also trying to keep support for the "legacy" project structures (at least for now) -- which does append those when it copies files. We'll look into this, because clearly it is not a good thing.

Eric Haas (Oct 07 2020 at 01:49):

add the all important id paremeter to the this documentation

image.png

Eric Haas (Oct 07 2020 at 01:49):

so instead would be...

dependencies:
  hl7.fhir.us.core:
    uri: http://hl7.org/fhir/us/core
    version: 3.1.1
    id: uscore
  hl7.fhir.us.davinci-hrex:
    uri: http://hl7.org/fhir/us/davinci-hrex
    version: 0.2.0
    id: hrex
parameters:

Eric Haas (Oct 07 2020 at 02:47):

actually this is wrong as well should be the 'Identity of the IG that this depends on' so

dependencies:
  hl7.fhir.us.core:
    uri: 'http://hl7.org/fhir/us/core/ImplementationGuide/hl7.fhir.us.core'
    version: 3.1.1
    id: uscore
  hl7.fhir.us.davinci-hrex:
    uri: 'http://hl7.org/fhir/us/davinci-hrex/ImplementationGuide/hl7.fhir.us.davinci-hrex'
    version: 0.2.0
    id: hrex

Shovan Roy (Oct 07 2020 at 04:54):

Chris Moesel said:

Hi Shovan Roy -- thanks for the feedback. Currently there is not a way to make SUSHI distribute the files into separate folders. We decided to put them all into resources for simplicity, but if you and others prefer them being split, that is good information to know. This is something we could revisit -- but it will also require additional changes to the base template. If you don't mind me asking, why do you like them split? Is it just to make it easier for you to personally review them?

Thanks @Chris Moesel @Jose Costa Teixeira @Eric Haas ,

The earlier structure was more helpful for us for the following reasons:

1. We are expecting that the number of profiles for our IG may grow up to ~250 profiles (SD) having ~400 samples and ~25 terminology artifacts. Having dedicated folder for each type of artifacts helps to manage things.
2. We get mix of profile contributors. Some of them are more comfortable to edit profiles on a existing UI based tool and some uses sushi. and there are people who are working only on samples. So keeping everything in one folder may be bit hectic.
   Thus, I was wondering if introduction of flag can be wished in 'sushi-config.yaml' which can instruct the engine to generate the older structure under 'fsh-generated' if enabled.

Jose Costa Teixeira (Oct 07 2020 at 06:39):

My current view on this is: you do need to structure the files and folders, but what you structure is the input content, you don't need to structure files that are not authored.

Jose Costa Teixeira (Oct 07 2020 at 06:40):

the publisher already supports this by using certain folders.

Jose Costa Teixeira (Oct 07 2020 at 06:41):

sushi can already support several files - which is what I use: I have a few files e.g. vaccination-profiles.fsh, vaccination-examples-fsh, vaccination-vocabulary.fsh.

Jose Costa Teixeira (Oct 07 2020 at 06:43):

in a later step, it would be more convenient for me to control more the file names, but as I think of it, the advantage would be only to simplify my copy-paste, because I won't commit the fsh-generated folder, only the input/... folders, including input/fsh

Jose Costa Teixeira (Oct 07 2020 at 06:43):

IIRC, you can add folders under input/fsh

Chris Moesel (Oct 07 2020 at 12:18):

@Eric Haas -- thanks for reporting on the issue in the documentation. Regarding id, we'll add that to the doc. Re: url, we've got it right in our code and in our tests, but we got it wrong in the FSH School documentation. We'll fix that. One thing to note, however, is that in most cases you don't need to put dependency URLs in the config. If you don't include it, SUSHI will look up the package by packageId and version then extract the url from the IG resource in the package (if it exists).

Chris Moesel (Oct 07 2020 at 12:32):

@Shovan Roy -- thanks for providing more detail. We will consider adding a control parameter in the config to affect how/where the files are output. We do, however, recognize that while there's value in choice there is also value in consistency -- so I can't promise we'll actually do it, but I can promise we'll revisit it. If others feel strongly about this, we'd love to hear from them.

As for your current situation, as others have noted, in typical SUSHI use cases, you should not be managing anything in the output fsh-generated folder, because it is not the source. The .fsh files are the source -- and SUSHI allows you to arrange those in very flexible ways. In fact, directly managing files in fsh-generated will not work because SUSHI clears all files in fsh-generated before it runs. Any changes you make will be deleted every time SUSHI runs.

For people who build and edit the resources using other tools (not FSH/SUSHI), they should be managing those files in the standard locations under input (which allow for multiple predefined subfolders). SUSHI will never overwrite those files, but it will read them so that .fsh files can reference them. If you want to create resources using FSH but then edit them in other tools, you'll need to copy them from fsh-generated/resources to input/* and then remove or comment out the FSH definition in your source. It's not advised to have multiple concurrent "gold source"s for the same resource.

I hope that helps, but if I've misunderstood your workflow, feel free to correct me!

Eric Haas (Oct 07 2020 at 12:59):

thanks, I Know but the ids are helpful for managing links in Jekyll. they shorten site variables considerably

Eric Haas (Oct 07 2020 at 13:00):

so its worth it to spend a little more effort in the configuration

Mark Kramer (Oct 07 2020 at 13:48):

Please join our discussion tomorrow, Thursday October 8 at 9-10 am Eastern US time. We will be focusing on the SUSHI 1.0 release.
Join Microsoft Teams Meeting
+1 540-492-5664 United States, Roanoke (Toll)
Conference ID: 889 990 654#

Last updated: Apr 12 2022 at 19:14 UTC