Stream: IG creation
Topic: Selecting SUSHI Versions in AutoBuild
Chris Moesel (Oct 02 2020 at 13:03):
Hi @Josh Mandel and @Grahame Grieve -- we plan to release SUSHI 1.0.0 Beta 1 today. This beta supports a new (and I think better) approach to IG Publisher integration. We are releasing it as beta to ensure it gets some revs w/ various projects before we force it on everyone. As a beta, it won't be tagged as the "latest" release in NPM (as that's reserved for stable releases) -- so npm install -g fsh-sushi will still install the 0.16.0 version. I think this means the autobuild will continue to run SUSHI 0.16.0, which is probably appropriate for most people.
My question (sorry it took so long to get here) is if there is a way for autobuild to run the beta version of SUSHI for projects that request/require it? IIRC, Grahame added a sushi.ini capability to affect the timeout when running SUSHI. Could this also be used to indicate a specific tag or version to use when installing SUSHI?
Chris Moesel (Oct 02 2020 at 13:05):
This would be helpful because in order to fully leverage the beta, authors need to change their project structure -- and if they do that, SUSHI 0.16.0 will fail to build it.
Josh Mandel (Oct 02 2020 at 15:14):
I think this means the autobuild will continue to run SUSHI 0.16.0, which is probably appropriate for most people.
That's correct.
Josh Mandel (Oct 02 2020 at 15:15):
IRC, Grahame added a sushi.ini capability to affect the timeout when running SUSHI. Could this also be used to indicate a specific tag or version to use when installing SUSHI?
We can define something like this, for sure. Let's see...
Josh Mandel (Oct 02 2020 at 15:16):
https://github.com/FHIR/auto-ig-builder/blob/c19e9050b1613d34d2116e7418d11001496cdc02/images/ig-build/builder/builder.py#L42 is where the autobuilder installs sushi. This is done after the target repo is git-cloned, so we'd have access to any files from the repo at this point.
Josh Mandel (Oct 02 2020 at 15:17):
I haven't see documentation for the sushi.ini format, and a Zulip search only turns up this thread. Is it a Windows-format INI file that would be compatible with https://docs.python.org/3/library/configparser.html ?
Josh Mandel (Oct 02 2020 at 15:20):
Basically, if you can write down the logic for:
1) Where to find a sushi.ini file relative to the repository root (if it exists)
2) Where, inside the sushi.ini file to find information about which fsh-sushi version to install
3) How to translate this into an npm install... command, and what default to use if none is present
... we should be able to make this work.
Chris Moesel (Oct 02 2020 at 16:20):
Cool. I only stumbled on sushi.ini the other day myself. It seems that @Grahame Grieve invented it. So... I actually have no more details than you. Grahame, can you confirm that sushi.ini follows the Windows INI format (like ig.ini, I assume)? And can you confirm it goes in repository root? If you can confirm those, I can take care of #2 and #3 above.
Grahame Grieve (Oct 02 2020 at 19:53):
it's actually fsh.ini, and it's a windows ini file, and it's current format is
[FSH]
timeout={msec}
Grahame Grieve (Oct 02 2020 at 19:53):
this is only a question for auto-build?
Grahame Grieve (Oct 02 2020 at 19:54):
no. it's most definitely a question elsewhere. What's the expectation for when we do release builds?
Chris Moesel (Oct 02 2020 at 20:06):
this is only a question for auto-build?
no. it's most definitely a question elsewhere.
On local builds it always uses whatever version of SUSHI the user has installed on their system. So locally, the user already has control over what version of SUSHI is used. To ensure no confusion, the property could be autobuild-sushi-version={version|tag}.
What's the expectation for when we do release builds?
I guess we have some choices. Do you want to allow people to have flexibility of SUSHI version for release builds? Or do you want to require the most recent official release? If you're concerned about people tacking their projects to old SUSHI builds then we could intentionally limit the fsh.ini capability. Instead of allowing a specific version, we could only allow the current released version or the most recent prerelease version (e.g., only support an optional allow-sushi-prerelease=true).
Chris Moesel (Oct 02 2020 at 20:07):
My main concern is allowing users to move to cutting edge prerelease versions if they want/need to.
Grahame Grieve (Oct 02 2020 at 21:52):
well, I'm not sure. I don't want to have to chop and change manually on the release build, but the version of sushi isn't really anything to do with the quality directly
Chris Moesel (Oct 05 2020 at 21:21):
@Grahame Grieve -- would you be ok w/ us introducing this capability in fsh.ini so that people can control the SUSHI used in auto-build? I would expect this would mainly be used only temporarily in IGs -- either to leverage a feature in a beta version of SUSHI or in rare cases to use an older version of SUSHI if/when the current SUSHI has a show-stopper bug for a particular IG. Whether or not the publication team allows for IGs to use a non-current (or beta) version of SUSHI for final publication seems more like a policy matter -- and it's something that can be checked easily enough by looking at the fsh.ini (i.e., you can choose to outright reject officially publishing any IG that has overridden the SUSHI version).
Grahame Grieve (Oct 05 2020 at 21:53):
I don't know quite enough about npm here. is it possible to have multiple different versions installed, and to use a nominated version when invoking sushi?
Chris Moesel (Oct 05 2020 at 22:25):
NPM doesn't allow you to globally install multiple versions of the same package. The closest they come to something like that is npx. The npx command (which is packaged with NPM) is a "package runner".
If you do npx fsh-sushi it locally installs and invokes the latest stable release of SUSHI. If you do npx fsh-sushi@0.16.0 it locally installs and invokes the specific version of SUSHI (in this case 0.16.0). If you do npx fsh-sushi@beta it installs the version of SUSHI tagged as beta. Etc. For example npx fsh-sushi@0.16.0 . -o build is the same as running sushi . -o build if SUSHI 0.16.0 were installed globally.
Grahame Grieve (Oct 05 2020 at 22:26):
I think I have to do this - I'm one of a number of people who build lots of IGs, and have already been tripped up by not having the correct version. If there's more than one version in play....
Chris Moesel (Oct 05 2020 at 22:27):
If you were to called npx fsh-sushi in the IG Publisher instead of just sushi, however, the end result is that the globally installed sushi would never be used. Maybe that's a good thing, I don't know -- but it is different than what happens now. In that case, it would probably be pretty important for users to be able to control what version of SUSHI the publisher invokes when it calls npx.
Grahame Grieve (Oct 05 2020 at 22:28):
well, what I'm thinking is that I'll look in fsh.ini to decide whether I'm going to run it directly or run a particular version
Chris Moesel (Oct 05 2020 at 22:29):
Ah. OK. That would be backward compatible then. So if you see something in fsh.ini, you use npx fsh-sushi@${version-from-ini} otherwise you just invoke sushi and use whatever is on the path.
Chris Moesel (Oct 05 2020 at 22:32):
Then the thing in fsh.ini controls the version of SUSHI that is used no matter where the publisher is invoked (i.e., locally and in @Josh Mandel's Auto-Builder (and Josh shouldn't need to do anything special). In that case, we'd probably want to call the ini property something simpler and have something like sushi-version=0.16.0.
Grahame Grieve (Oct 05 2020 at 22:37):
what versions can I test this with right now?
Jose Costa Teixeira (Oct 06 2020 at 05:36):
v0.16.0 and v1.0.0-beta.1 are considerably different. v1.0.0-beta.1 uses the new folder structure (no ig-data, instead input is in input/fsh and output in fsh-generated
Jose Costa Teixeira (Oct 06 2020 at 05:37):
I can create a fork of the sample IG with this new config if you want to use that for testing.
Chris Moesel (Oct 06 2020 at 12:32):
Sorry, had to step out -- but @Jose Costa Teixeira is right. To use v0.16.0:
npx fsh-sushi@0.16.0 .
and to use the beta:
npx fsh-sushi@1.0.0-beta.1 .
or more simply, using the beta tag:
npx fsh-sushi@beta .
You'll notice that I just used . as the argument (indicating current folder). That assumes you are running it from the root folder of the IG. In this case, it will find the input path (whether it's fsh or input/fsh) and also correctly choose the appropriate output path. I noticed that the IG Publisher explicitly passes in the input and output paths, but this is not necessary.
If you're just looking to test this locally, to make sure it really uses the right versions, you can just invoke it with the -v flag to have it only print out the version.
One thing to note, it will redownload and install (in a temporary location) SUSHI on every invocation, so depending on how long that takes, you may want to consider bumping the default timeout by 10 or 15 seconds to accommodate it.
Grahame Grieve (Oct 07 2020 at 00:22):
ok this works fine. It'll be in the next release
Grahame Grieve (Oct 07 2020 at 00:22):
[FSH]
version=1.0.0-beta.1
Grahame Grieve (Oct 07 2020 at 00:23):
I'm not yet going to require that IGs using sushi set this before making a publication request, but I'm planning to do that onec we see how it works
Chris Moesel (Oct 07 2020 at 12:34):
Would it be possible to call that sushi-version rather than just version? I'm afraid that version is too ambiguous and there might be confusion between the version of FSH and the version of SUSHI (which are definitely two different things).
Grahame Grieve (Oct 07 2020 at 23:35):
ok. done
Chris Moesel (Oct 07 2020 at 23:50):
Thank you!
Chris Moesel (Oct 09 2020 at 14:03):
Hi @Grahame Grieve -- I just tried this out and it did not work. I tried to determine if maybe the IG Publisher hasn't been released yet since you fixed it -- but I don't see your fix at all in the IG Publisher commits on GitHub. Am I looking in the wrong place?
Grahame Grieve (Oct 09 2020 at 18:29):
I haven't committed yet. getting there
Chris Moesel (Oct 14 2020 at 14:49):
Hi @Grahame Grieve -- do you have an ETA on this? There are a few IG authors who have already converted to the beta structure, but can't leverage auto-builds again until this feature is in place. They're wondering when it will be available. If it's still not close, would it be possible to push just a commit supporting the new fsh.ini stuff? (I'm assuming the stuff holding it up is not related to fsh.ini at all).
Grahame Grieve (Oct 14 2020 at 19:13):
a few hours
Grahame Grieve (Oct 14 2020 at 21:24):
@Chris Moesel released
Chris Moesel (Oct 15 2020 at 00:09):
Confirmed to be working. For those of you who need to use the SUSHI 1.0.0 beta releases, I recommend the following fsh.ini config:
[FSH]
sushi-version=beta
See the thread on the shorthand stream for more details.
Grahame Grieve (Oct 15 2020 at 00:31):
in the future, IGs built by Sushi will be required to specify the correct version of FSH to run before submitting for publication
Last updated: Apr 12 2022 at 19:14 UTC
