FHIR Chat · Custom template folder · shorthand

Stream: shorthand

Topic: Custom template folder

Noemi Deppenwiese (Jul 02 2020 at 13:21):

I changed an existing IG to use SUSHI for newly added profiles, but I ran into an Issue with my custom template. I have it in a folder "mytemplate" next to the "input" in the "ig-data" folder. I assumed when using the ig-data folder everything not somehow processed by SUSHI would simply be copied to the "build" directory. But only the "input" folder is in the "build" directory. Is this behaviour intended or a bug? Can I get SUSHI to copy the folder? I would preferr not to having to pre-propulate the "build" folder to avoid confusion...

Jose Costa Teixeira (Jul 02 2020 at 13:55):

I don't use that folder structure, and it seems the sushi folder structure and integration may cause you issues there.

Noemi Deppenwiese (Jul 02 2020 at 14:02):

What folder structure do use use? does it work?

Jose Costa Teixeira (Jul 02 2020 at 14:38):

why not have the template on the parent above the fsh folder?

Jose Costa Teixeira (Jul 02 2020 at 14:45):

image.png

Jose Costa Teixeira (Jul 02 2020 at 14:46):

this works for me (but I don't want sushi to take over my IG, whereas I think you probably want that)

Jose Costa Teixeira (Jul 02 2020 at 14:48):

others can advise what is the best way to keep things intact, but the approach above makes sure that fsh/sushi is simply another way to provide some content to the IG. I haven't tried the new config file approach, so i hope I am not misleading you

Jose Costa Teixeira (Jul 02 2020 at 14:49):

@Chris Moesel has better advice

Noemi Deppenwiese (Jul 02 2020 at 14:50):

Yes, that works for the template folder, but then SUSHI fails to auto-generate the pages section in the IG (which was one of the reasons i wanted to switch to the new config-approach). So, two mutually exclusive workarounds for two maybe-bugs....
Still, thanks for the advice!

Chris Moesel (Jul 02 2020 at 16:35):

@Noemi Deppenwiese -- we're currently looking at how we copy things during the build and will be considering a more liberal approach. As it is now, SUSHI only copies over certain folders -- so custom template folders are not copied. We recognize that it may make more sense to just copy over everything (even things SUSHI doesn't recognize / understand). Tagging @Nick Freiter since he is doing this work.

For now, I think you could apply a hybrid strategy of what Jose recommends. I think you can put your custom template folder at the same level as the fsh folder (so outside the fsh folder) -- but still put your pages in fsh/ig-data/pagecontent (or pages). Then you still benefit from the advantages of the SUSHI processing on your pages, but the IG Publisher will also be able to find your custom template.

Jose Costa Teixeira (Jul 02 2020 at 16:47):

@Chris Moesel I am trying to have something that works but now sushi is asking me for a config.yaml - my IG wass already building OK, so how can I tell sushi to not change anything, just produce the profiles?

Jose Costa Teixeira (Jul 02 2020 at 16:51):

concretely, why do I now need a config.yaml file, I just want sushi to generate the profiles - can we remove that need? Even the minimal file is already defining properties that are defined elsewhere, so that seems a bit invasive.

Jean Duteau (Jul 02 2020 at 16:58):

Here's the minimum config.yaml that will just use your FSH files. config.yaml

As far as I know, you just have to update canonical to be the canonical that you want for your guide, as SUSHI needs that to generate proper URLs for the various artifacts.

Jean Duteau (Jul 02 2020 at 17:00):

since we have FSHOnly set to true, many of the other variables are irrelevant.

@Chris Moesel Although I think that Jose's use of SUSHI is not in line with the rest of us, he does have a point. If I set FSHOnly to true, many of the other currently mandatory variables are ignored, so we might need to change the required nature of them.

Jose Costa Teixeira (Jul 02 2020 at 17:01):

thanks Jean

Jose Costa Teixeira (Jul 02 2020 at 17:02):

that still duplicates data from what I have in my ig.ini, etc. files

Jean Duteau (Jul 02 2020 at 17:03):

Sure, as I said, you need those variables or SUSHI will complain, but other than canonical, it can be anything since it appears to get ignored when FSHOnly is set to true.

Jose Costa Teixeira (Jul 02 2020 at 17:04):

And please don't mind but it's not that my "use of sushi is not in line with the rest of us". I am doing this to actually add sushi to the standard IG structure.
I do expect sushi to do its primary task - generating profiles from shorthand, without trying to step in my IG process

Jean Duteau (Jul 02 2020 at 17:04):

that's not sushi's primary task

Jean Duteau (Jul 02 2020 at 17:05):

it hasn't been for awhile.

Jose Costa Teixeira (Jul 02 2020 at 17:05):

ok. Sushi's primary task is not to parse shorthand?

Jean Duteau (Jul 02 2020 at 17:05):

no, SUSHI's primary task is to create an IG from FSH files.

Jean Duteau (Jul 02 2020 at 17:06):

as part of that, it does parse shorthand, but there is a lot of other stuff that it does and the required variables in the config reflects that primary use case.

Jean Duteau (Jul 02 2020 at 17:07):

what you really need is the IGPublisher to be understand FSH files natively and generate FHIR artifacts from those.

Jose Costa Teixeira (Jul 02 2020 at 17:07):

I was not aware of that decision but anyway - my need was just to align the 2 "worlds" - those that use tools like Forge etc to generate profiles, and trifolia and other IG editors, and those that only use sushi.

Jean Duteau (Jul 02 2020 at 17:09):

But I agree that the team can make it slightly better for you by identifying the proper set of config variables that are needed by SUSHI if someone wants a FSHOnly use of the compiler.

Jose Costa Teixeira (Jul 02 2020 at 17:09):

Jean Duteau said:

what you really need is the IGPublisher to be understand FSH files natively and generate FHIR artifacts from those.

no, what we need is a shorthand parser that does its job, handles logical models, can do reverse parsing... and leave the IG stuff for those tools, or provide an additional optional path

Jose Costa Teixeira (Jul 02 2020 at 17:10):

I thought there was an idea to split the functionalities in sushi between fsh / IG.

Jose Costa Teixeira (Jul 02 2020 at 17:11):

Which would be very good because I am using the standard toolset, building that up, and I don't want every discussion to be that I have to choose

Jean Duteau (Jul 02 2020 at 17:41):

Jean Duteau said:

since we have FSHOnly set to true, many of the other variables are irrelevant.

Chris Moesel Although I think that Jose's use of SUSHI is not in line with the rest of us, he does have a point. If I set FSHOnly to true, many of the other currently mandatory variables are ignored, so we might need to change the required nature of them.

Jose and I had a private chat about this and we agree with the following changes (if you'll humour us in telling you what to do!):
Short-term: have FSHOnly = true in the config mean that only the config variables are needed to do FSH parsing are required
Long-term: have FSHOnly = true mean that it assumes that the input/* files exist and that it can find any paths and such that it needs by looking at those instead of needing any config at all. This might also mean changing FSHOnly into a command-line flag to SUSHI.

Chris Moesel (Jul 02 2020 at 18:31):

I guess it's good to have passionate users. ;-)

So... I agree that if FSHOnly is set, we should be more lax about required properties in the config.yaml. The addition of the FSHOnly flag was late-breaking, so if SUSHI requires properties that aren't actually necessary in that case, then it's an oversight (not intentional).

We do plan to separate the Implementation Guide features out of SUSHI and into their own library/tool (which can be invoked as part of the SUSHI process as well). We think it makes sense to separate the concerns, and in an ideal world, we hope to see IG features move into the IG Publisher to the extent that our IG tool is no longer necessary.

I'm not sure if I fully understand the proposed long-term solution. Is it that if FSHOnly is used, SUSHI looks for an ImplementationGuide resource in input/ and uses that to dig out the canonical? One thing we've talked about is having a classic FHIR project structure, with no fsh folder, and putting your .fsh files in input/* just like your non-FSH files. SUSHI then finds and processes these and puts the output into a special temp/sushi folder that the IG Publisher knows to look for -- so it will include the SUSHI outputs in its processing. This would eliminate a lot of the confusion that the fsh/ig-data creates.

Jean Duteau (Jul 02 2020 at 18:36):

Chris Moesel said:

I'm not sure if I fully understand the proposed long-term solution. Is it that if FSHOnly is used, SUSHI looks for an ImplementationGuide resource in input/ and uses that to dig out the canonical? One thing we've talked about is having a classic FHIR project structure, with no fsh folder, and putting your .fsh files in input/* just like your non-FSH files. SUSHI then finds and processes these and puts the output into a special temp/sushi folder that the IG Publisher knows to look for -- so it will include the SUSHI outputs in its processing. This would eliminate a lot of the confusion that the fsh/ig-data creates.

Yes, I'm not sure what variables it needs to output the various FHIR artifacts from the fsh files, but the idea is that it would simply use the ImplementationGuide resource (or the package.json or ig.ini or whatever files that I don't care about now that I use SUSHI) to get what it needs. Your even longer term solution of real integration with the IG Publisher would be even better for Jose and who want straight FSH parsing.

Jose Costa Teixeira (Jul 02 2020 at 19:57):

this is the minimal config that made it work:

id: example.fhir.uv.myig (1)
canonical: http://hl7.org/fhir/uv/myig (2)
version: 0.2.0 (1)
fhirVersion: 4.0.1 (1)
FSHOnly: true
name: YourComputableMyIGNameHere (1)
status: draft (1)

Jose Costa Teixeira (Jul 02 2020 at 19:57):

(1) is an attribute that I copied from the ig.xml
(2) comes from the package-list.json

Jose Costa Teixeira (Jul 02 2020 at 19:58):

Chris Moesel said:

Is it that if FSHOnly is used, SUSHI looks for an ImplementationGuide resource in input/ and uses that to dig out the canonical?

yes

Jose Costa Teixeira (Jul 02 2020 at 20:02):

Chris Moesel said:

One thing we've talked about is having a classic FHIR project structure, with no fsh folder, and putting your .fsh files in input/* just like your non-FSH files. SUSHI then finds and processes these and puts the output into a special temp/sushi folder that the IG Publisher knows to look for -- so it will include the SUSHI outputs in its processing.

If you put the fish and the meat together in the same tank (which may be OK but I'd keep an eye on the naming convention and folder structure for that), why do you put the parsed output in a temp folder and not back in the tank?

Chris Moesel (Jul 03 2020 at 00:08):

why do you put the parsed output in a temp folder and not back in the tank?

We want to keep the true source files completely separate from the output files -- otherwise things get confusing, it's difficult to know what to check in and what not to check in, and it's difficult to do a true clean build since you would need to delete the output files one-by-one since they're mixed in with source files.

Chris Moesel (Jul 03 2020 at 00:13):

We're aware of how those config params map to the other files; because SUSHI uses the config to generate the other files. But I think we're on the same page that most of them should not be necessary for a FSH-only build. The two caveats are: (1) I don't think an ImplementationGuide is guaranteed to be present in every project, and (2) package-list.json is only required for HL7 projects. So if there is no existing ImplementationGuide or package-list.json (or package.json) then SUSHI would need to get the canonical URL from somewhere (and that somewhere would probably be the config.yaml).

Chris Moesel (Jul 03 2020 at 00:15):

But... the suggestion is noted. It seems like a reasonable mode to support in SUSHI (the no-config mode that uses existing files if it can find them). We'll definitely look into it.

Jose Costa Teixeira (Jul 03 2020 at 05:06):

Chris Moesel said:

So if there is no existing ImplementationGuide or package-list.json (or package.json) then SUSHI would need to get the canonical URL from somewhere (and that somewhere would probably be the config.yaml).

The idea is that if sushi can get content from those files, it should, instead of failing. For now, it fails.

Jose Costa Teixeira (Jul 03 2020 at 05:08):

if I have an IG and want to add shorthand sources to it, i don't want to have to add files with duplicate data.

Jose Costa Teixeira (Jul 06 2020 at 20:32):

Chris Moesel said:

So... I agree that if FSHOnly is set, we should be more lax about required properties in the config.yaml. The addition of the FSHOnly flag was late-breaking, so if SUSHI requires properties that aren't actually necessary in that case, then it's an oversight (not intentional).

@Chris Moesel when should we expect this to be fixed?

Chris Moesel (Jul 06 2020 at 20:50):

We work in 2-week sprints. We just started our current sprint on Thursday and it's fully planned out already. So... we're probably looking at the next sprint for this, which means it would be released in 2-4 weeks.

Noemi Deppenwiese (Jul 07 2020 at 08:48):

Chris Moesel said:

Noemi Deppenwiese -- we're currently looking at how we copy things during the build and will be considering a more liberal approach. As it is now, SUSHI only copies over certain folders -- so custom template folders are not copied. We recognize that it may make more sense to just copy over everything (even things SUSHI doesn't recognize / understand). Tagging Nick Freiter since he is doing this work.

For now, I think you could apply a hybrid strategy of what Jose recommends. I think you can put your custom template folder at the same level as the fsh folder (so outside the fsh folder) -- but still put your pages in fsh/ig-data/pagecontent (or pages). Then you still benefit from the advantages of the SUSHI processing on your pages, but the IG Publisher will also be able to find your custom template.

I finally got around to try that approach. The path needs to be fsh/ig-data/input/pages (without ig-data or input it does not work). Now I have both the poupulated pages section and the template folder, so thanks!
(I now get some error messages about how some of my my non-FSH resources have no id which was no problem with the IG Publisher before, but that can be solved.)

Chris Moesel (Jul 07 2020 at 11:31):

Thanks for following up, @Noemi Deppenwiese. I'm glad to hear it is working! Sorry about the confusion on pages -- I didn't write up the full path because I already had done that for pagecontent and I guess I was getting lazy! As for the id requirement -- yes, SUSHI currently requires ids -- as it uses them to organize the resources internally as well as in its file-naming convention when it exports them. That's probably something we can look at in the future.

John Moehrke (Jul 10 2020 at 12:35):

so i understand that sushi is going to get the canonical URL from the input/ implementationGuide. What element in there? It does not seem to be following the .url element.

Nick Freiter (Jul 10 2020 at 13:12):

SUSHI gets the canonical URL from the config.yaml file, and the URL is specified using a canonical field.

John Moehrke (Jul 10 2020 at 13:16):

so i must have a config.yaml? is that new?

David Pyke (Jul 10 2020 at 13:17):

That was new in 0.13 and should have been created for you

John Moehrke (Jul 10 2020 at 13:19):

how is it created for me? And if it is created for me, then how do I get the right caonical in there?

John Moehrke (Jul 10 2020 at 13:21):

where is config.yaml documented?

John Moehrke (Jul 10 2020 at 13:25):

what I can find on this config.yaml indicates it is only when there is no canonical in the implementationguide. I have one there, it just doesn't seem to be used consistently

John Moehrke (Jul 10 2020 at 13:28):

so I should not need a config.yaml. right?

David Pyke (Jul 10 2020 at 13:29):

https://build.fhir.org/ig/HL7/fhir-shorthand/sushi.html#configuration-file

Nick Freiter (Jul 10 2020 at 13:29):

http://build.fhir.org/ig/HL7/fhir-shorthand/branches/master/sushi.html#configuration-file has documentation, and https://github.com/FHIR/sushi/releases/tag/v0.13.0 contains some additional documentation.

Assuming you have an existing FSH project with a working SUSHI configuration, SUSHI should create a config.yaml file from that configuration when you run an updated version of SUSHI on that project. SUSHI does not get its configuration from input/ImplementationGuide. You will always need a config.yaml, since that is how SUSHI is configured.

David Pyke (Jul 10 2020 at 13:29):

Mine is in my fsh directory

John Moehrke (Jul 10 2020 at 13:30):

well, sushi did not make a config.yaml

John Moehrke (Jul 10 2020 at 13:30):

and the documentation says that it will get caonical from the ig...

John Moehrke (Jul 10 2020 at 13:30):

so I am confused

John Moehrke (Jul 10 2020 at 13:32):

is this a replacement for the package.json in the fsh directory?

John Moehrke (Jul 10 2020 at 13:34):

I did find the bad canonical url in the package.json

Nick Freiter (Jul 10 2020 at 13:34):

Yes, SUSHI used to get its configuration from package.json, now it is configured by config.yaml. This is a sample repo that shows how a project using SUSHI might be structured with config.yaml https://github.com/standardhealth/fsh-devdays-exercise. It's difficult to say why SUSHI did not make a config.yaml for you, perhaps you are using an older version of SUSHI?

John Moehrke (Jul 10 2020 at 13:34):

so wondering if I continue to manage the package.json, or move this into config.yaml

John Moehrke (Jul 10 2020 at 13:35):

I updated my sushi today. is that fresh enough?

John Moehrke (Jul 10 2020 at 13:35):

So I understand I should switch to config.yaml, and remove package.json from my fsh tank. right?

Nick Freiter (Jul 10 2020 at 13:40):

That should be plenty fresh, config.yaml is the way to configure SUSHI going forward, and package.json is deprecated. SUSHI is supposed to generate config.yaml from your existing package.json, so I'd like to know why that isn't happening since it seems to be a bug, any extra info you can give us to reproduce that is appreciated.

John Moehrke (Jul 10 2020 at 13:48):

I updated my repo, and you can look at it. https://github.com/JohnMoehrke/MHD-fsh

John Moehrke (Jul 10 2020 at 13:56):

so I just removed package.json, and created what I think is the right config.yaml... now sushi fails with an error that seems to indicate it must find a package.json

John Moehrke (Jul 10 2020 at 13:56):

Run Sushi on C:\Users\john.moehrke\Git\MHD-fsh (00:00.0072)
Sushi: info Running SUSHI v0.12.6 (00:01.0318)
Sushi: info Current FSH tank conforms to an IG Publisher context. Output will be adjusted accordingly. (00:01.0320)
Sushi: error No package.json in FSH definition folder. (00:01.0323)

John Moehrke (Jul 10 2020 at 14:04):

I restored my original package.json,.. and it is building... so sushi does still require package.json

Nick Freiter (Jul 10 2020 at 14:21):

Sushi: info Running SUSHI v0.12.6

It seems your SUSHI hasn't updated, could you again try npm install -g fsh-sushi? And then run sushi -v to see which version you have? The current version of SUSHI is 0.14.0, and 0.12.6 is an old version that uses package.json

John Moehrke (Jul 10 2020 at 14:23):

now at v0.14.0

Nick Freiter (Jul 10 2020 at 14:24):

Ok, then v0.14.0 version should work with a config.yaml file.

John Moehrke (Jul 10 2020 at 14:38):

getting further. but now I have my implementationGuide file (xml) and sushi creates an implementationGuide file (json).

Last updated: Apr 12 2022 at 19:14 UTC