FHIR Chat · dev dependencies · shorthand

Stream: shorthand

Topic: dev dependencies

David Hay (Apr 20 2020 at 02:53):

An issue came up when editing another projects fsh tank. If there is a dependency in package.json labelled #dev, then sushi looks in the local cache to see if the package is there - failing if it can't find it. @Grahame Grieve has suggested that in this scenario sushi should 'fall back' to #current, and attempt to download it...

Grahame Grieve (Apr 20 2020 at 02:56):

that's what #dev is documented to mean

Chris Moesel (Apr 20 2020 at 13:23):

OK, just found the referenced doc: https://confluence.hl7.org/pages/viewpage.action?pageId=35718627#IGPublisherDocumentation-DependencyList

dev: use the latest local build of the IG. in the absence of a local build, this falls back to using the current version

We'll look at this in SUSHI. Thank you!

Chris Moesel (Apr 20 2020 at 13:26):

OK, tracking it in SUSHI as #374.

Nick Freiter (Apr 24 2020 at 16:15):

So while trying to add support for automatically falling back to #current, we've discovered a difference in how SUSHI handles #current and how the IG Publisher seems to. With SUSHI, if a dependency is specified as#current, then even if the desired package already exists in the local FHIR cache, it is overwritten with the newest version found on https://build.fhir.org/ig/qas.json, even if that newest version is exactly the same as what is already cached. We do not try to determine if the package has been updated or not, we just always overwrite.

From some experimenting, it seems like when the IG Publisher sees a dependency that is specified as #current, it does not automatically overwrite that package if it already exists in the local cache. @Grahame Grieve , we were wondering if you could tell us how the IG Publisher determines what to do in this situation. If the IG Publisher uses a certain method for determining when a locally cached #current package needs to be overwritten, and when it does not, we would like to try to match that method in SUSHI.

Grahame Grieve (Apr 24 2020 at 22:21):

being Australian, I am super concerned about bandwidth. (I've currently used 120G of my 80GB allowance 1/3 of the way through the month).

Grahame Grieve (Apr 24 2020 at 22:30):

pseudo code is :

  NpmPackage npm = loadFromLocalCache(id, "current");
  Map<PackageId, URL> ciList = loadfrom("https://build.fhir.org/ig/qas.json");
  Json json = fetchJson(ciList.get(npm.name()) + "/package.manifest.json");
  String currDate = json.getString("date");
  String packDate = npm.date();
  if (currDate.equals(packDate))
      return p;
  else
      fetchFromCIBuild(id);

Nick Freiter (Apr 27 2020 at 12:22):

Great, thank you for the help! We will look to update that in SUSHI to help save some bandwidth. One more question, where does npm.date() pull the date from? Does it come from the package.json of the relevant package?

Nick Freiter (Jun 02 2020 at 14:43):

@Grahame Grieve we have noticed an odd discrepancy between loading the package hl7.fhir.us.core#3.1.0 with the IG Publisher vs with SUSHI. When SUSHI loads this package, it downloads it from packages.fhir.org/hl7.fhir.us.core/3.1.0, and the folder that gets decompressed and added to the FHIR cache has this structure:
image.png

The above is the same structure you see if you visit packages.fhir.org/hl7.fhir.us.core/3.1.0 and decompress the downloaded package manually.

But if we instead use the IG Publisher to add this package to the cache, the folder that gets decompressed and added to the cache has this structure:
image.png

And the example, openapi, other, and xml folders are nested within that package folder. This is the format we would expect based on the specification of FHIR NPM packages. So we are wondering:

1. Is the IG Publisher getting that package from packages.fhir.org?
2. If yes, is the IG Publisher doing anything special to change the format of the package downloaded from packages.fhir.org?
3. If yes, do you expect it could be an issue if SUSHI does not change the format of the package in a similar way?

Thank you for any insight!

Grahame Grieve (Jun 02 2020 at 21:30):

There's history here. When we first wrote the NPM package specification, we said that these other folders would be peers of the package directory. And that's what you're seeing

Grahame Grieve (Jun 02 2020 at 21:33):

but in November last year, at DevDays in Amsterdam, it was pointed out in the Packages session that this is not conformant and they have to sub-folders of package not peers. It's too late to fix previously published packages but all new packages will have the folders as sub-folders. For consistency, those peer folders are turned into sub-folders when the package is installed into the local cache.

Grahame Grieve (Jun 02 2020 at 21:34):

I think it probably does matter that things are consistent. It might not matter to the java tools - they probably ignore the difference when reading the installed package too... not sure. but other tools will eventually run into this. probably. Or possibly, at least

Nick Freiter (Jun 03 2020 at 14:50):

Interesting, thank you for the detailed explanation!

Chris Moesel (Jun 17 2020 at 16:29):

BTW -- @Nick Freiter implemented the package restructuring in SUSHI here: https://github.com/FHIR/sushi/pull/482

Last updated: Apr 12 2022 at 19:14 UTC