FHIR Chat · Dependencies · shorthand

Stream: shorthand

Topic: Dependencies

Max Körlinge (Mar 16 2021 at 08:54):

Hi everyone! I am getting started using shorthand for a profile/IG that has to extend the german base profile. I have found it in the hl7 registry, the top version points at https://ig.fhir.de/basisprofile-de/0.2.30/ - but I cannot get Sushi to find the dependency. I have tried various ways of specifying the id, the url, the version (also tried the r4 version), but I cannot get it to work. Does anyone know how to specify this dependency correctly in config.yml?

Chris Moesel (Mar 16 2021 at 12:41):

Hi @Max Körlinge -- I was able to find that IG in the FHIR Registry here: https://registry.fhir.org/package/basisprofil.de|0.2.30

A few things to note about it:

• There is actually a new version (0.2.40)
• It is based on FHIR STU3

Since it is based on FHIR STU3, and SUSHI only supports FHIR R4, I cannot recommend using it in your own IG. The version mismatch would cause problems. It appears that SUSHI does allow you to indicate it as a dependency, but I think this is a bug in SUSHI -- as it should reject IGs that are from a different major version of FHIR from the IG that depends on them.

We don't currently have any plans to support FHIR STU3 in SUSHI, so unfortunately I don't think you'll be able to depend on this IG until it has an R4 version -- or you'll need to develop your own IG using a different tool that supports FHIR STU3.

Max Körlinge (Mar 16 2021 at 14:33):

Thank you @Chris Moesel ! I have located a R4 alpha version to use instead at https://simplifier.net/Basisprofil-DE-R4/~introduction. And Sushi compiles without errors with

dependencies:
  de.basisprofil.r4:
     uri: http://fhir.de/ImplementationGuide/basisprofil-de-r4
     version: 1.0.0-alpha9

It really was some detective work to find the URI - going to the package files and finding the implementation guide. Is there supposed to be a simpler way to find out which URI to depend on for packages?
I haven't tried to extend anything from it yet though so it remains to be seen if that works.

Chris Moesel (Mar 16 2021 at 14:58):

Thanks for the update, @Max Körlinge. I'm glad that there is an R4 version!

When SUSHI downloads the package, it tries to find the IG URL itself by looking through any ImplementationGuide resources found in the package. When it does that, it is looking specifically for an ImplementationGuide with a packageId value matching the package id in your declared dependencies. It looks like Simplifier does not populate the ImplementationGuide.packageId field -- so it doesn't recognize the IG file as a match. @Ward Weistra -- is there a reason Simplifier does not populate packageId or is this something you could potentially support in the future?

Max Körlinge (Mar 16 2021 at 15:20):

Is the URI I input in the dependencies section correct then? To refer directly to the implementation guide uri?

As you can tell, I am a beginner with FSH and FHIR as well :sweat_smile: just a developer who has to define a profile based on the german national one for our startup. FSH appealed to me because it is code, which I'm used to working with. If I want to create a profile based on one of the german profiles - take for example this one https://simplifier.net/Basisprofil-DE-R4/VitalSignDE/~overview - how would I reference that in the "Parent" section of my profile?

Profile: MyObservation
Parent: http://fhir.de/StructureDefinition/observation-de-vitalsign

or

Profile: MyObservation
Parent:  VitalSignDE

both give me Structure Definition ... is missing snapshot. Snapshot is required for import error. I am not sure if this means that the dependency failed to import correctly or if it means I am putting the wrong thing in "Parent" - or if I am completely doing the wrong thing here

If there is some documentation that deal with this usecase (extending/profiling from another IG/profile) please direct me there, I haven't been able to find any through google :/

Chris Moesel (Mar 16 2021 at 16:00):

Either approach (full canonical URL or formal name) is valid for indicating the parent. The issue you are running into here is separate... The dependency package that was downloaded apparently does not include the snapshot section in its profiles -- and SUSHI requires the snapshot in order to process the parent definition correctly. I'm not sure the best way forward here, so I will tag a few people and then present one potential approach.

• @Grahame Grieve -- At one point a while ago, you suggested that you might build in an option on the validator to update a package in the FHIR cache w/ snapshots. I don't see any documentation about this, so I'm assuming that has not been implemented, but wanted to double-check.

• @Ward Weistra -- Can you remind me what the current thinking is on this topic? Is Simplifier considering publishing w/ snapshots? Do you have any suggestions for how users can use SUSHI w/ published IGs having no snapshots?

Here is a workaround you can use if you'd like. It's clunky, but it should work:

1. Go to the specific package you want on Simplifier (from the project, click "Packages" tab, then click the specific package)
2. On the top right, click "Download" and then "Download snapshots" (note: you must be logged in)
3. This will download a .tgz file representing the package (e.g., de.basisprofile.r4-1.0.0-alpha9.tgz)
4. Find the corresponding folder in your FHIR cache. For example, on a mac, this is likely something like: ~/.fhir/packages/de.basisprofil.r4#1.0.0-alpha9. If the folder doesn't exists, create it. If the folder does exist, delete its contents.
5. Unzip the .tgz into that folder in your FHIR cache
6. You've just replaced the package in your cache w/ one that has snapshots, so SUSHI should work now.

That's not ideal. I know. If someone has any better approaches, I'd love to hear them!

Max Körlinge (Mar 16 2021 at 18:18):

Thanks a lot Chris! For my usecase, since I will only be using one external profile, the workaround will do just fine (I will try it tomorrow in practice). If I have time I will try to write some blog post on this process when it's done for posterity =)

Chris Moesel (Mar 16 2021 at 20:27):

@Max Körlinge -- that would be splendid! SUSHI and FSH are still young -- so we're still working through things like this, especially as it relates to interaction w/ other tools in the ecosystem... Let me know if you run into any issues trying this out.

Ward Weistra (Mar 17 2021 at 16:43):

@Chris Moesel :wave:
On pointing to the package from the IG resource: Yes, we look at this next sprint. In our publishing process this wasn't straightforward, but indeed it's mandatory in R4.

On snapshots in packages: Indeed, we discussed this before and I see why this would be useful for non-Java and .NET based tooling. From a technical standpoint we've never been a fan of it because it bloats the size of packages by quite a bit and snapshot generation between the stacks isn't consistent enough to rely on each other's snapshots (although it's being worked on).

On top of that there's also practical impossibilities I learned today: Packages can specify wildcard dependencies, like this Nictiz project is specifying a dependency on the latest version of the 2.0.x range of versions of the nictiz.fhir.nl.stu3.zib2017 package. That makes the snapshots of the package necessarily a function of time. Including snapshots would require to update the package every time a new dependency package is added to the range, which would change the NPM API checksum and break the promise of never changing packages.

The 'Download with Snapshots' option you referred to, which we provide on any package page, will calculate the snapshots at that time. And perhaps we can provide that via a non-NPM (because changing packages with changing checksums) API, although that might not in an acceptable time frame.
Would it be an option to add a Java or .NET based 'calculate the snapshots we need' step to sushi? I understand you'd prefer to prevent that dependency if possible, but I'm not sure if there's a good alternative.

Chris Moesel (Mar 17 2021 at 20:12):

Hi @Ward Weistra. Oh, that is complex, isn't it? I also was not aware of wildcard dependencies like you describe (although current is a type of wildcard I guess).

I have to admit, though, I was under the impression that a given package version represented the state of affairs as of when that package was released. While a wildcard dependency is fine during the development cycle, I'd suggest that all dependencies must be fixed to specific versions at publication time. Otherwise it seems to me we could land in a real interoperability mess. What does it mean for a system to support Acme 1.2.0 if the technical definition of Acme 1.2.0 can change at any moment? FHIR packages aren't software -- they are formal specifications; so it seems to me that introducing the possibility of unpredictable variation in a published release is a real problem.

That's probably a whole different conversation that should be had somewhere else though. If it's impractical for you to publish packages w/ snapshots I think we can try to find some other creative solutions. One could be to invoke the Java tools to generate the snapshots (although that raises some other questions). Another could be to add a switch to SUSHI to automate steps 4-6 above for any tgz package (or to support loading a tgzpackage dependency in place). Another would be Grahame's idea of supporting a mode on the validator that populates snapshots for a package already in the FHIR cache...

Ward Weistra (Mar 18 2021 at 08:53):

@Chris Moesel I understand your worry with wildcards, but it also solves entire trees of dependencies updating only for trivial changes. In the spec the standard wildcards of NPM have already been limited to the patch version number only. So, changes SHOULD only be trivial and not impact conformance. (another reason why we should understand and apply good semver versioning in the FHIR community for packages...)

Some way to have the Java (validator?) or .NET (via Simplifier.net download or Firely Terminal) stack calculate the correct snapshots at a certain timepoint would indeed work. Such a tool should also look at the wildcard dependencies and update any dependency packages if needed and (re)calculate the snapshots if needed.

Simone Heckmann (Mar 18 2021 at 10:21):

@Max Körlinge if you have any questions regading the german base profiles, please join us in https://chat.fhir.org/#narrow/stream/179183-german-(d-a-ch)

Max Körlinge (Mar 18 2021 at 12:33):

thanks @Simone Heckmann

Patrick Werner (Apr 06 2021 at 14:20):

Chris Moesel said:

Max Körlinge -- that would be splendid! SUSHI and FSH are still young -- so we're still working through things like this, especially as it relates to interaction w/ other tools in the ecosystem... Let me know if you run into any issues trying this out.

This is still an issue i'd like to support to fix. @Chris Moesel Would a java cli tool help you which does the snapshot generation for you help?

Patrick Werner (Apr 06 2021 at 14:20):

If yes i can probably help.

Patrick Werner (Apr 06 2021 at 14:23):

This tool then would snapshot a package and then..? I don't like the idea of overwriting a package in the fhir cache too much, also i don't like creating snapshots every run.
What about adding the snapshots to the sushi project?

Chris Moesel (Apr 06 2021 at 18:52):

Ideally there would be a JS snapshot generator, but I'm not aware of any. @Brian Kaney -- your shop hasn't done anything like that, has it? Assuming not then we could potentially launch a Java jar to generate a snapshot -- but I do have a few concerns there:

• We'd have to rely on the user having Java installed correctly.
• We'd also need to figure out where the jar comes from (do we package it in our npm package, download it on the fly, etc).
• As you noted, we'd probably want some caching mechanism so we don't run snapshot generation every time.
• It wouldn't work on web-based versions of SUSHI (e.g., FSH Online).

But those all may be worth it in order to have a working solution for folks using packages without snapshots.

Another potential approach would be to use a web service that does snapshot generation on the fly, but we're not in the position to host such a thing, so someone else would need to provide that.

And one more approach would be to better automate some (or all???) of the steps I noted above (assuming that 90+% of the snapshot-less IGs are coming from Simplifier).

Oliver Egger (Apr 06 2021 at 19:16):

We'd also need to figure out where the jar comes from (do we package it in our npm package, download it on the fly, etc).

The FHIR Java validator has the capability to generate snapshots for a profile with the -snapshot option.

Patrick Werner (Apr 07 2021 at 07:40):

Chris Moesel said:

And one more approach would be to better automate some (or all???) of the steps I noted above (assuming that 90+% of the snapshot-less IGs are coming from Simplifier).

Yes the easiest solution would be that all FHIR packages are containing Snapshots.

Patrick Werner (Apr 07 2021 at 07:41):

Second easiest would be a webservice as a proxy to the fhir package registry, snapshoting and caching the packages

Ward Weistra (Apr 07 2021 at 08:24):

The .fhir FHIR cache folder currently contains a packages directory. Tools not being able to generate snapshots themselves could perhaps use a sibling packages-with-snapshots-added folder with the same setup? Not a source of original truth, but a derived folder from the content of packages.
That could then be populated with the Java Validator or Firely Terminal. This would be cleanest given that range dependencies will update and that snapshot generation is not a fixed art yet.

We can also look into making the 'download package with snapshots' endpoint from Simplifier available as an API, but that would then probably be a non-NPM API with some free rate limit per user. I wouldn't want two competing FHIR Package Registry NPM APIs. And preferably not any packages in packages-with-snapshots-added that are not in packages, because then other tools can't necessarily reuse an already downloaded package.

Patrick Werner (Apr 07 2021 at 08:27):

i like the idea of the packages-with-snapshots-addedfolder next to the package folder

Jose Costa Teixeira (Apr 07 2021 at 08:32):

Are the current packages in the cache without snapshots?

Patrick Werner (Apr 07 2021 at 08:33):

mixed, All IG Publisher packages are including snapshot, Simplifier doesn't

Patrick Werner (Apr 07 2021 at 08:33):

The FHIR package spec doesn't state anything on snapshot vs no snapshot

Jose Costa Teixeira (Apr 07 2021 at 08:42):

ok, thanks

Janaka Peiris (Jun 01 2021 at 17:49):

(deleted)

Jose Costa Teixeira (Jun 02 2021 at 14:42):

What can be done here?

Jose Costa Teixeira (Jun 02 2021 at 14:44):

I think I also like the idea of a sibling, but could it be under the same package structure?
i.e. could a package contain two sets of files, one with and the other without the snapshots? or is that really another package (which makes it simpler to implement, but I guess harder to maintain)

Jose Costa Teixeira (Jun 02 2021 at 14:45):

alternatively:
could Simplifier open the downloads of snapshot-enabled packages (no login needed), and sushi always get those?

Jose Costa Teixeira (Jun 02 2021 at 14:46):

Or we could include an extension in the IG dependsOn to indicate forcing the snapshots.
So that IG-publisher / sushi is in control of which package to download

Jose Costa Teixeira (Jun 03 2021 at 10:46):

@Robin Bosman

Patrick Werner (Jun 24 2021 at 15:12):

as i am still needing a solution i'll probably going to implement a small java application, checking the dependencies in the sushi config yaml, downloading them and checking if snapshots are available and building snapshots for the ones that don't have them.

Patrick Werner (Jun 24 2021 at 15:14):

to not overwrite the package cache i would like to add them to a folder siblingpackages-with-snapshots-addedfolder of the package folder.

Patrick Werner (Jun 24 2021 at 15:16):

@Chris Moesel could sushi be configured to first look into the sibling folder for needed dependencies before doing the rest of the package loading?

Jose Costa Teixeira (Jun 24 2021 at 16:25):

Thank you Patrick. It would be good if a proper solution is found for this (I'd use your patch if you make it available, but I just want to keep pointing to where it hurts)

Patrick Werner (Jun 24 2021 at 16:51):

it won't be a patch, rather a pre-processor. Without sushi looking into the proposed folder i would have to overwrite packages in the cache

Patrick Werner (Jun 24 2021 at 16:52):

If sushi would search in another folder first (or only, handled by the settings) we won't have to overwrite existing cached packages

Chris Moesel (Jun 24 2021 at 16:52):

That's an interesting idea, @Patrick Werner. It sounds like a reasonable approach, but let me discuss it w/ the team before I commit to anything! The one wrinkle that still exists is that these IGs would still fail in an autobuild environment since the autobuild won't have the special snapshot folder in the cache.

Patrick Werner (Jun 24 2021 at 16:53):

then the autobuild would habe to pre-process the sushi file as well...?

Chris Moesel (Jun 24 2021 at 16:54):

Right. Which means we'd need to tie in Josh or someone else who knows how to work w/ the autobuild too.

Patrick Werner (Jun 24 2021 at 16:54):

@Ward Weistra what do you think about including snapshots into packages with only stable dependeny trees? You wouldn't have to watch for the whole tree all the time, as everything has to be declared to a specific version

Patrick Werner (Jun 24 2021 at 17:03):

Chris Moesel said:

Right. Which means we'd need to tie in Josh or someone else who knows how to work w/ the autobuild too.

not necessarily right now, the autobuilder is already failing with non-snapshoted packages. So no change here.

Jose Costa Teixeira (Jun 24 2021 at 21:14):

We can do something in the scripts.
We could also do it on the github workflows, but I think this should be deeper in the tools (i.e. inside sushi which is where the complaints show, or inside the autobuild that the github workflows rely on).
I did ask for a solution in the publisher itself

Ward Weistra (Jun 25 2021 at 11:06):

@Patrick Werner @Chris Moesel @Jose Costa Teixeira I'll admit I went back and forth on this, but think we should start including them, even on the ranged dependencies. Talking to @Ewout Kramer I learned snapshots were always meant from the FHIR spec as a convenience cache so not every tool would need to understand the snapshot generation logic.
Downstream tools (incl sushi) should keep in mind that snapshots are basically a cache, however, which they are allowed to recreate when they have reason to suspect the cache to be outdated (e.g. on ranged dependencies or changes to snapshot generation logic in a toolstack).

Our backlog is long however, so I don't have a time estimation of when we get around to this exactly. Something to balance with other customer needs.
I also expect only to start adding snapshots for newly created package versions (cause we cannot change existing package versions), and will not necessarily check incoming packages via the package feeds for this. So packages without snapshots won't be history any time soon and it would be good to give users a way to use them. Even if only explaining the procedure how to generate or download from Simplifier with snapshots and use those.

Chris Moesel (Jun 25 2021 at 12:53):

@Ward Weistra -- That's great! I think that will help everything go more smoothly in the future (at least for us!). Thanks for thinking through all the implications for this. Currently the download from Simplifier requires a log-in. I don't suppose it's any quicker/easier to expose that as a more standard API, is it? If so, it might be worth considering as well, since that would take care of both past and future packages.

Jose Costa Teixeira (Jul 10 2021 at 14:06):

Kind of related to this, how does fsh get the packages / where does it get it from when the IG depends on them?
I have an IG and I want to do dependsOn, where I indicate the package id and url. Does that url / package id need to be in the registry? (which registry?)

Jose Costa Teixeira (Jul 10 2021 at 14:07):

Use scenario - I have a "core" ig and I want to dependOn it for my purpose-specific IGs. the IG is published, with packages and redirects accessible, but Sushi doesn't get the package

Chris Moesel (Jul 12 2021 at 18:13):

(deleted)

Chris Moesel (Jul 12 2021 at 18:19):

There are a few places we load packages from:

• First, we try to find it in the local FHIR cache by id and version. If it's there, that is what we use.
• If the package version is current, we grab the qas.json file from build.fhir.org and use that to try to find the last build.
• Otherwise, we try to find it on packages.fhir.org via http://packages.fhir.org/${packageId}/${version}.
• If it's not there and the package is R4B or R5, we try packages2.fhir.org via http://packages2.fhir.org/packages/${packageId}/${version}

David Hay (Jul 12 2021 at 18:36):

I noticed that if the dependency is set to 'current' but it has only been built locally then it won't find it. You have to set it to 'dev'

Chris Moesel (Jul 12 2021 at 20:59):

Yeah, current specifically means the current build from the CI server. dev means the latest local build. If there is no dev locally, it will fall back to current. But not the other way around.

Jose Costa Teixeira (Jul 12 2021 at 22:14):

Is this documented somewhere? (sorry if i missed it). If not, should it?

Ward Weistra (Jul 16 2021 at 13:59):

@Chris Moesel To make the 'related questions' chain even longer: How do you resolve dependencies from packages? Just downloading every package, opening up the package.json, reading the dependencies and dowload those all the way down?

Ward Weistra (Jul 16 2021 at 14:03):

Also, on the packages with snapshots front: Our first approach to providing everyone with a solution is to implement a 'generate snapshots for these packages/my dependencies/my entire package cache' as a (free) feature in Firely Terminal. Snapshotting individual resources was already there and the size increase and processing burden are taken up by the consumer' machine. Testing the first release with this internally now.

Nick Freiter (Jul 19 2021 at 13:45):

@Ward Weistra, unless I'm misunderstanding, you're asking how we handle dependencies of dependencies? So for example if package A is an explicit dependency, but then package A depends on package B. In SUSHI we don't actually need package B, since if the definitions in A have their snapshot (which we require), that is all of the information that SUSHI needs. So SUSHI only resolves explicitly listed dependencies.

Ward Weistra (Jul 19 2021 at 16:32):

@Nick Freiter Thanks! That explains. @Martijn Harthoorn wondered, since it apparently isn't trivial :wink:

Chris Moesel (Jul 20 2021 at 16:40):

Yeah, we don't do that at all. I think there may be a few cases that perhaps it could cause an issue (in rare cases where we need to walk the full inheritance tree and a parent is from a transitive dependency) -- but those cases are very rare and have not yet come up in practice. In that case, we'd probably just suggest explicitly listing the transitive dependency -- at least for now. It may be something we need to reconsider in the future though.

Josh Mandel (Aug 04 2021 at 19:26):

Chris Moesel said:

There are a few places we load packages from:

• First, we try to find it in the local FHIR cache by id and version. If it's there, that is what we use.
• If the package version is current, we grab the qas.json file from build.fhir.org and use that to try to find the last build.
• Otherwise, we try to find it on packages.fhir.org via http://packages.fhir.org/${packageId}/${version}.
• If it's not there and the package is R4B or R5, we try packages2.fhir.org via http://packages2.fhir.org/packages/${packageId}/${version}

Can someone clarify how qas.json is used? What branches of a given dependency are counted toward matching the "current"? Is this limited to main/master, or is there external logic? Otherwise there's a risk taht updates to some random branch on a project might be counted as "the most current" version of a dependency.

Josh Mandel (Aug 04 2021 at 19:28):

(Reading this in light of recent challenges from #IG creation -- I totally didn't realize that qas.json was being used as a source of truth for anything.)

Nick Freiter (Aug 04 2021 at 20:48):

For SUSHI, the qas.json file is used to find all packages that match the package-id of the desired package. Those packages are then sorted by date, and the newest package is selected. There is no checking based on branch, which does definitely seem like an issue as you pointed out. For my understanding, does the autobuild only publish packages based on the master or main branch of a repo? Or is that something that could vary?

Nick Freiter (Aug 04 2021 at 20:49):

Here is the pseudocode from the IG Publisher that we used to implement our package resolution in SUSHI: https://chat.fhir.org/#narrow/stream/215610-shorthand/topic/dev.20dependencies/near/195252120. Our goal is to match the IG Pub behavior.

Josh Mandel (Aug 04 2021 at 22:51):

Thanks Nick! Yeah, we'll need to figure out how to pin these down. Discussion continuing on that thread on #IG creation .

John Moehrke (Dec 16 2021 at 18:04):

I tried to build a sushi project that has dependencies on some IHE published IGs. they could not be found. I used the same coordinates that I see @Oliver Egger was able to use in a classic xml based IG. In sushi-config.yaml

dependencies:
  ihe.mhd.fhir:
    id: ihemhdfhir
    uri: http://profiles.ihe.net/ITI/MHD/ImplementationGuide/ihe.mhd.fhir
    version: 4.0.3
  IHE.ITI.PDQm:
    id: ihepdqm
    uri: http://profiles.ihe.net/ITI/PDQm/ImplementationGuide/IHE.ITI.PDQm
    version: 2.3.1

sushi error about these is could not be loaded locally or from the FHIR package registry.

These are registered in the old and new systems.

John Moehrke (Dec 16 2021 at 18:16):

never mind... chair-to-keyboard interface problem.. Need to use formal release numbers, not current CI build. duh

Thomas Tveit Rosenlund (Jan 14 2022 at 08:29):

I still have a problem. After changing my dependency in the sushi-config.yaml to this:

dependencies:
    hl7.fhir.no.basis: 2.1.1

I finally get to load the package into the sushi/ig-publisher workflow. However soon after it complains:

Publishing Content Failed: The package 'hl7.fhir.no.basis' has no entry on the current build server

Looks like it only tries to resolve on build.fhir.org. Any ideas? The package is built and published using SIMPLIFIER. @Chris Moesel

Chris Moesel (Jan 14 2022 at 14:36):

I think that error message comes from the IG Publisher (not SUSHI), so I'm not sure what it means. I have not seen it before. @Grahame Grieve - what does Publishing Content Failed: The package 'hl7.fhir.no.basis' has no entry on the current build server mean, and how can Thomas fix it?

Thomas Tveit Rosenlund (Jan 14 2022 at 16:23):

Chris Moesel said:

I think that error message comes from the IG Publisher (not SUSHI), so I'm not sure what it means. I have not seen it before. Grahame Grieve - what does Publishing Content Failed: The package 'hl7.fhir.no.basis' has no entry on the current build server mean, and how can Thomas fix it?

I think you are right @Chris Moesel . My mistake.

The problem, I think, is that I use the hl7 provided docker build image in my Github action script (this works fine for building things that are not dependent on other packages):

- name: 🏃‍♂️ Run IG Publisher
        uses: docker://hl7fhir/ig-publisher-base:latest
        with:
          args: java -jar publisher.jar publisher -ig ${{ env.IG }}/ig.ini -publish https://hl7norway.github.io/no-domain-vitalsigns/

And I suppose that is the same thing running on the build.fhir.org. The problem is that this image only recognizes packages that are built and distributed using that workflow. This is my current interpretation of the error.

So how to solve it. I suppose I can do two tings.

1. Build another version of my docker build image, that contains the no-basis package and see if that works.
2. Import all dependencies in the Github action and run the IG publisher there (not using docker image), in that case I can probably import the dependencies the normal way using an npm install command. The flip side would be that I use a lot of Github runtime just setting up my IG publisher build environment on the virtual ubuntu node running the script.

Chris Moesel (Jan 14 2022 at 16:30):

@Josh Mandel - do you have any insight/guidance here?

Grahame Grieve (Jan 14 2022 at 17:41):

I don't know why this is happening. hl7.fhir.no.basis#2.1.1 is found on packages.fhir.org, so it should just automatically resolve

Grahame Grieve (Jan 14 2022 at 17:41):

do you get any messages in the output related to hl7.fhir.no.basis ?

Thomas Tveit Rosenlund (Jan 14 2022 at 18:48):

Grahame Grieve said:

do you get any messages in the output related to hl7.fhir.no.basis ?

Actually yes. Sushi says it downloads the package OK:

Sushi: info  Running SUSHI v2.2.6 (implements FHIR Shorthand specification v1.2.0) (00:00.0527)
Sushi: info  Arguments:                                                          (00:00.0528)
Sushi: info    --out /github/workspace/VitalSigns                                (00:00.0530)
Sushi: info    /github/workspace/VitalSigns                                      (00:00.0531)
Sushi: info  Using configuration file: /github/workspace/VitalSigns/sushi-config.yaml (00:00.0534)
Sushi: info  Importing FSH text...                                               (00:00.0545)
Sushi: info  Preprocessed 91 documents with 25 aliases.                          (00:00.0900)
Sushi: info  Imported 76 definitions and 14 instances.                           (00:00.0963)
Sushi: info  Checking local cache for hl7.fhir.no.basis#2.1.1...                 (00:00.0966)
Sushi: info  Did not find hl7.fhir.no.basis#2.1.1 in local cache.                (00:00.0967)
Sushi: info  Downloading hl7.fhir.no.basis#2.1.1...                              (00:00.0969)
Sushi: info  Checking local cache for hl7.fhir.r4.core#4.0.1...                  (00:00.0985)
Sushi: info  Did not find hl7.fhir.r4.core#4.0.1 in local cache.                 (00:00.0988)
Sushi: info  Downloading hl7.fhir.r4.core#4.0.1...                               (00:00.0990)
Sushi: info  Downloaded hl7.fhir.no.basis#2.1.1                                  (00:01.0662)
Sushi: info  Loaded package hl7.fhir.no.basis#2.1.1                              (00:01.0697)
Sushi: info  Downloaded hl7.fhir.r4.core#4.0.1                                   (00:04.0113)
Sushi: info  Loaded package hl7.fhir.r4.core#4.0.1                               (00:05.0833)
Sushi: info  Converting FSH to FHIR resources...                                 (00:05.0948)
Sushi: info  Converted 51 FHIR StructureDefinitions.                             (00:13.0352)

Thomas Tveit Rosenlund (Jan 14 2022 at 18:50):

But the IG-publisher build process still can't find it.
The complete log is here:

https://github.com/HL7Norway/no-domain/runs/4815170074?check_suite_focus=true

Grahame Grieve (Jan 14 2022 at 19:02):

I'm not sure why you don't use the normal structure for an IG, so that it runs using the usual infrastructure

Grahame Grieve (Jan 14 2022 at 19:08):

anyway that means I can't investigate and don't know how to reproduce

Chris Moesel (Jan 14 2022 at 19:49):

@Grahame Grieve -- I was able to reproduce this with just the standard publisher. Here are the steps:

1. git clone git@github.com:HL7Norway/no-domain.git or git clone https://github.com/HL7Norway/no-domain.git
2. cd no-domain
3. git checkout feature/dipsfsh2-workflow
4. cd VitalSigns
5. sh _genonce.sh or _genonce.bat

Things to note:

• Repo is https://github.com/HL7Norway/no-domain
• Must use feature/dipsfsh2-workflow branch (master branch is not setup for publisher)
• Must change directory to VitalSigns (root dir is not right directory format)

Chris Moesel (Jan 14 2022 at 19:54):

It happens after Load Package hl7.terminology.r4#3.0.0.

Grahame Grieve (Jan 14 2022 at 19:54):

thanks. I don't have time to investigate this stuff

Chris Moesel (Jan 14 2022 at 20:02):

@Grahame Grieve and @Thomas Tveit Rosenlund - FOUND IT. Thomas, your ig.ini does not point to the ImplemenationGuide JSON that SUSHI creates. It points to a different one. So SUSHI and the IG Publisher are using two different configs. While the SUSHI config points to hl7.fhir.no.basis#2.1.1, the IG resource that ig.ini points to specifies hl7.fhir.no.basis#dev (link). When the publisher can't find a local dev copy of a dependency, it falls back to looking for it on the build server. So it all makes sense and is working as expected.

You need to either edit your ImplementationGuide JSON so the dependency is correct or you need to have ig.ini point to the ImplementationGuide JSON that SUSHI generates.

Grahame Grieve (Jan 14 2022 at 20:04):

good work. that would be why I recommend a standard set up

Chris Moesel (Jan 14 2022 at 20:10):

@Thomas Tveit Rosenlund - I'd recommend either removing your custom ImplementationGuide JSON and just using the SUSHI-generated one, OR removing sushi-config.yaml so that SUSHI uses the ImplementationGuide JSON you've provided. Either way, you end up with a single config, which is good.

Thomas Tveit Rosenlund (Jan 15 2022 at 08:04):

Chris Moesel said:

Thomas Tveit Rosenlund - I'd recommend either removing your custom ImplementationGuide JSON and just using the SUSHI-generated one, OR removing sushi-config.yaml so that SUSHI uses the ImplementationGuide JSON you've provided. Either way, you end up with a single config, which is good.

Excactly. Thanks for the help @Chris Moesel you just saved me hours of work, don't know if I would have figured that one out.

The problem is that I don't normaly use a separate ImplementationGuide resource, but the project responsible for these profiles did, and when I tried to put this one into the automated build process we are testing for the HL7 Norway base and domain profiles, it would not work out of the box. The error messages did not make much sense to me as SUSHI clearly downloaded the package in question. I will simply remove the IG-resource and make my configuration in the sushi-yaml file the way I use to.

Thanks again!

Thomas Tveit Rosenlund (Jan 15 2022 at 08:27):

And just like that, it works beautiflully. Have to fix the sushi-yaml to incorporate the rest of the documentation pages though.

https://github.com/HL7Norway/no-domain/runs/4825786507?check_suite_focus=true

Chris Moesel (Jan 18 2022 at 14:01):

Yes, you'll need to move at least the page listing and any parameters values over to sushi-config.yaml. The full sushi-config.yaml is documented here.

Last updated: Apr 12 2022 at 19:14 UTC