FHIR Chat · NPM Package File · IG creation

Stream: IG creation

Topic: NPM Package File

Grahame Grieve (Mar 22 2018 at 13:00):

@Ewout Kramer , working on the NPM package file for the IGs. Here's a draft package.json for the spec itself:

Grahame Grieve (Mar 22 2018 at 13:04):

{
  "name": "@fhir/fhir",
  "version" : "3.2.0",
  "description": "Base FHIR specification Package",
  "license": "CC0-1.0",
  "author": {
    "name": "HL7 Inc"
  },
  "files": [
    "igpack.zip"
  ]
}

Grahame Grieve (Mar 22 2018 at 13:08):

this is the package.json for a 1st derivative IG:

Grahame Grieve (Mar 22 2018 at 13:18):

{
  "name": "@fhir/au-base",
  "version": "3.0.1",
  "description": "Implementation Guide Template",
  "license": "CC0-1.0",
  "author": {
    "name": "HL7 Australia"
  },
  "contributors": [
    {
      "name": "HL7 Australia Patient Administration Working Group",
      "url": "http://confluence.hl7australia.com/display/PA/Patient+Administration+Work+Group"
    },
    {
      "name": "HL7 Australia Orders and Observations Working Group",
      "url": "https://confluence.hl7australia.com/display/OO/Orders+and+Observations+Work+Group"
    },
    {
      "name": "HL7 Australia Medications Working Group",
      "url": "https://confluence.hl7australia.com/display/MED/Medications+Work+Group"
    }
  ],
  "files": [
    "validation.pack"
  ],
  "dependencies": {
    "@fhir/fhir": "3.0.1"
  }
}

Grahame Grieve (Mar 22 2018 at 13:19):

comments?

Ewout Kramer (Mar 22 2018 at 14:27):

Accidentally, we have been discussing packaging over the past few days, since the whole Simplifier team is now present and packaging is in our next design phase for Simplifier. I know @Martijn Harthoorn already had some feedback on the wiki page.

Grahame Grieve (Mar 22 2018 at 21:27):

.. but no comment yet...?

Grahame Grieve (Mar 22 2018 at 21:30):

here's a package.json for a 2nd derivative IG:

{
  "name": "@fhir/au-pd",
  "version": "3.0.1",
  "description": "Implementation Guide Template",
  "license": "CC0-1.0",
  "author": {
    "name": "HL7 Australia"
  },
  "contributors": [
    {
      "name": "HL7 Australia Patient Administration Working Group",
      "url": "http://confluence.hl7australia.com/display/PA/Patient+Administration+Work+Group"
    },
    {
      "name": "Secure Messaging Technical Working Group",
      "url": "https://collaborate.digitalhealth.gov.au/display/SMT/Secure+Messaging+TWG+Home"
    }
  ],
  "files": [
    "validation.pack"
  ],
  "dependencies": {
    "@fhir/fhir": "3.0.1",
    "@fhir/au-base": "3.0.1"
  }
}

Martijn Harthoorn (Mar 23 2018 at 14:22):

Is there a place where we can comment? Or do you want our comments by mail?

Martijn Harthoorn (Mar 23 2018 at 14:23):

Or I can type them here :)

Martijn Harthoorn (Mar 23 2018 at 14:26):

1.
Packages can be a very powerful adition to FHIR. And can make the use of easier to understand and use. But for it to be adopted by the FHIR community and implementers, a package system must be easy enough to use. The base choice for npm is understandable as it is one of the most widely used. However it will require each user to install nodejs and npm. clients. We suspect that half of the non technical users will quit before they have responded to the question "do you want to download the 32bit or the 64bit version".

In other words: the best scenario is where each FHIR tool that works with profiles and other conformance resources, is able to implement their own package client without too much problems. The community should be able to build package consumption libraries in various languages that can help add that client functionality to any conformance tooling app.

There for a FHIR package clients needs is just to be just a small subset of features that npm provides. We think it's unnecessary and probably even harmful for the client to do any javascript postprocessing.

Martijn Harthoorn (Mar 23 2018 at 14:27):

Npm features we think have to be excluded for a FHIR package client to be light and manageable:
- No package references to github tags, branches or commits.
- No javascript execution
- Maybe limit some of the very free versioning rules. References like wildcards, min max and multiple range probably lead to more confusion than help. Only Floating minor or bugfix versions are sufficient.

Martijn Harthoorn (Mar 23 2018 at 14:31):

2. Naming
Yes, naming needs to centralized to avoid collision, but I think it's not the best solution to have a central administrator. It will create a delay in peoples workflow and it will not scale if becomes succesful. And if we allow publication to npmjs.org, you cannot enforce it anyway, can you?
It might be more effective to have some clear guidelines. Like some package standards do enforce. NuGet now has the option to reserve prefixes.

Martijn Harthoorn (Mar 23 2018 at 14:36):

3. Scopes
Searching for scoped packages on npm is not trivial. Building dedicated fhir-npm package server seems doable. And with proper client tooling, it might be a lot easier to jump start a fhir-package ecosystem. - but this one is my personal opinion.

Martijn Harthoorn (Mar 23 2018 at 14:38):

4. It might be valuable to define some rules on how and where local fire packages are stored and cached. Because it is not trivial for a regular user to configure that in npm.

Martijn Harthoorn (Mar 23 2018 at 14:40):

And with a properly defined subset of npm (npm-light) we might standardize how fhir package tooling can deal with local storage and caching.

Eric Haas (Mar 23 2018 at 16:23):

What issues are the NPM packaging trying to solve? versioning? How is a published IG going to benefit from this?

Eric Haas (Mar 23 2018 at 16:24):

I assumed the IG is on the web and folks read it there and download the downloadable artifacts.

Martijn Harthoorn (Mar 23 2018 at 16:56):

The main improvements that packages solve is that it is a level of indirection for your references. This will make sure that a cluster of conformance resources refer to a consistent set of base resources.

Martijn Harthoorn (Mar 23 2018 at 16:57):

You can now refer to a package version instead of having to refer to a correct profile version in each reference.

Grahame Grieve (Mar 23 2018 at 19:29):

#1 - agree.
#2 - it's not like IGs roll over quickly - so I don't see the issue in requiring IGs published by HL7 or FHIR.org to be managed
#3 - I'm not sure what you mean by 'scoped packages'
#4 - what kind of rules could we define?

Martijn Harthoorn (Mar 26 2018 at 11:24):

#2. Ok didn't get that. I thought it was for all packages.
#3. Scoped as in npm scoped packages @fhir/ ... I guess if that too was only for HL7/FHIR.org, I don't seen an issue.

#4. The rules I can think of:
- Cache packages globally under ~/.fhir/packages or something similar.
- Do not use folder recursion: no dependencies under the current packages folder. Store all packages at the top level

It might be worth noting, that we foresee use cases where packages are consumed by profile authors before those packages are final. We've seen several cases already where projects started working together and depending on eachother while both still in draft.

Martijn Harthoorn (Mar 26 2018 at 11:26):

Regarding #4. Other rules could be
- Not to use javascript post processing.

Martijn Harthoorn (Mar 26 2018 at 11:30):

• No complex version references like "^1.3 || 2.x || >=3.5.0 || 5.0.0 - 7.2.3"

Martijn Harthoorn (Mar 26 2018 at 11:32):

I've tried to implement a npm client in C# with full the full version logic. It was doable, but it makes everything very complex and harder to manage.

Martijn Harthoorn (Mar 26 2018 at 11:33):

And it definitely does not satisfy the FHIR paradigm of moving complexity to the server ;)

Grahame Grieve (Mar 26 2018 at 11:35):

#2 I forgot to say.
#3 we'll have to see how it plays out...
#4
- cache packages globally... I don't know what that means unless it's a pure deployment thing
- no recursion - shudder. compleeetely agree. Same for javascript post-processing
- agree about version references. But what do we need? In eclipse land, I've always gotten by on >= x(, <= Y)?

Martijn Harthoorn (Mar 26 2018 at 11:37):

For sake of simplicity I would say the same rules that we have (or will have) for canonical references behind the pipe.

Grahame Grieve (Mar 26 2018 at 11:38):

bet we don't finish that simple. but I'm game to try

Martijn Harthoorn (Mar 26 2018 at 11:38):

:)

Martijn Harthoorn (Apr 09 2018 at 11:53):

@Grahame Grieve , we wrote a first draft text for Fhir packaging.
https://gist.github.com/mharthoorn/a52de54d1d010fb6c6e74f8a5d85fbc8

Grahame Grieve (Apr 09 2018 at 12:46):

ok I'll look at it. In fact, first scan... tarball.. really?

Martijn Harthoorn (Apr 09 2018 at 12:50):

I am not a fan of tarbal. But it's the npm package format.

Grahame Grieve (Apr 09 2018 at 14:32):

Should we really distribute implementation guides that way. it will make it slower and it won't be consumed by a regular package client. Maybe we need a regular not standardized zip for ig's. What is grahame's use case here?

Grahame Grieve (Apr 09 2018 at 14:33):

this is the core of the issue. I don't really have a use case; Ewout suggested that we use npm format for resolving dependencies, and for people to use when working with implementation guides... whatever that means. @Ewout Kramer - what does that mean

Grahame Grieve (Apr 09 2018 at 14:33):

but my pain point is resolving IG dependencies when building IGs

Martijn Harthoorn (Apr 09 2018 at 14:50):

But what you care about resolving is only the conformance resources. Not the static content, right? Or do you want to inject that too?

Grahame Grieve (Apr 09 2018 at 19:20):

I only care about the resources, yes

Ewout Kramer (Apr 13 2018 at 14:56):

So, maybe we should not distribute IG htmls etc. like this, it's just about a package.

Grahame Grieve (Apr 13 2018 at 20:59):

I'm happy if it's just the resources, and maybe a version indicator.

Grahame Grieve (Apr 25 2018 at 06:59):

ping to bring this back to life

Martijn Harthoorn (Apr 25 2018 at 10:33):

ping.

Martijn Harthoorn (Apr 25 2018 at 10:34):

We are currently building a C# FHIR packaging library - to allow consumption of packages by profile tools.

Martijn Harthoorn (Apr 25 2018 at 10:35):

We are also working on canonical url resolving in packages (forementioned global package cache)

Martijn Harthoorn (Apr 25 2018 at 10:36):

And we have build a command line tool to download, install and create packages specifically for FHIR.

Martijn Harthoorn (Apr 25 2018 at 10:37):

-- based on the preliminary choices that we've made sofar.

Grahame Grieve (Apr 25 2018 at 12:52):

ok, so I don't understand this comment:

Grahame Grieve (Apr 25 2018 at 12:52):

Should we really distribute implementation guides that way. it will make it slower and it won't be consumed by a regular package client. Maybe we need a regular not standardized zip for ig's. What is grahame's use case here?

Grahame Grieve (Apr 25 2018 at 12:52):

slower than what? why?

Grahame Grieve (Apr 25 2018 at 12:52):

what's a regular package client? what would it consume?

Grahame Grieve (Apr 25 2018 at 12:53):

what's the relationship between the name and the canonical URL?

Grahame Grieve (Apr 25 2018 at 12:53):

I think the package file needs the canonical URL somehow

Grahame Grieve (Apr 25 2018 at 12:54):

we cannot make semver mandatory. I'm happy to recommend it strongly

Grahame Grieve (Apr 25 2018 at 12:56):

license - I've been using https://spdx.org/licenses/ which is much more comprehensive than https://help.github.com/articles/licensing-a-repository/ and also I got that link from the npm documentation.

Grahame Grieve (Apr 25 2018 at 12:58):

master package dependency has to be versioned. And you haven't indicated quite what you think they should look like (other than inconsistency on the naming suggestions)

Martijn Harthoorn (Apr 25 2018 at 14:16):

A package with only the profiles is smaller (a lot) than the whole implementation guide (all pages, images, etc). And the primary use case that I see, the user doesn't need the source of the IG. he needs to find it rendered online somewhere.

and a smaller package is a faster download.

Martijn Harthoorn (Apr 25 2018 at 14:16):

The primary use case that I see for packages is automated profiles + dependencies distribution. A 'one click' install and you have all conformance resources in your scope - with the right version.

Martijn Harthoorn (Apr 25 2018 at 14:17):

-- a regular package client: any desktop tool that works with conformance resources.

Martijn Harthoorn (Apr 25 2018 at 14:18):

• relationship between a package name and a canonical URL: I have no idea yet. It might be valueable to establish a rule there, but I also think of meta packages (that only refer to other packages), where a relationship between a package name and a canonical url is not evident.

Martijn Harthoorn (Apr 25 2018 at 14:19):

semver: of course we cannot enforce it. But tooling can.

Martijn Harthoorn (Apr 25 2018 at 14:21):

-- license: sure. spdx looks complete :)

Martijn Harthoorn (Apr 25 2018 at 14:22):

"master package dependency has to be versioned" -- what do you mean here?

Martijn Harthoorn (Apr 25 2018 at 14:22):

We are experimenting with including an table in the package.json with a link from the canonical URL to the actual file.

Martijn Harthoorn (Apr 25 2018 at 14:23):

Do you want me to update the gist with what we discuss here?

Grahame Grieve (Apr 25 2018 at 21:39):

yes update the gist, though it needs to move to the wiki page at some stage.

Grahame Grieve (Apr 25 2018 at 21:40):

so I understand your responses about that note, but they don't explain the note itself

Grahame Grieve (Apr 25 2018 at 21:41):

some of my confusion is that I don't know, when you say 'profiles', whether you are stritctly referring to structure definitions, or just generally to conformance definitions. ome of your gist implies that you think that these things should be broken up (I strongly disagree with that). But I'm going to ignore the comment now.

Grahame Grieve (Apr 25 2018 at 21:42):

so let's say this: the package includes all the resources in the guide, in json (no xml in this context, since the package itself is json).

Grahame Grieve (Apr 25 2018 at 21:42):

" including an table in the package.json with a link from the canonical URL to the actual file" - which looks like?

Grahame Grieve (Apr 25 2018 at 21:56):

with regard to naming conventions, something like this:

hl7.fhir (main build)
hl7.us.core
hl7.au.base
hl7.nl.medmij
ihe.pix
ihe.mhd
[country].[company].[purpose]

Grahame Grieve (Apr 25 2018 at 21:59):

and on semver: my issue is a policy one: we should not force everyone to use semver. Nothing to do whether it's possible to enforce or not.

Martijn Harthoorn (Apr 26 2018 at 20:56):

If you do not enforce semver, you cannot use version wildcards or ranges. I am fine with that. But it will mean that you can only target 1 specific version. It's also a deviation from npm.

Martijn Harthoorn (Apr 26 2018 at 20:56):

@Ewout Kramer what is your opinion on this?

Grahame Grieve (Apr 26 2018 at 21:00):

you can still use version wildcards for some deviations, they just don't work well. I'm fine to strongly encourage semver, and can enforce it for HL7 and fhir.org implementation guides, but it won't be possible everywhere

Martijn Harthoorn (Apr 26 2018 at 21:02):

ok. I think I updated all comments.
https://gist.github.com/mharthoorn/a52de54d1d010fb6c6e74f8a5d85fbc8

Grahame Grieve (Apr 26 2018 at 21:04):

this gist approach is not working for me. Wiki or google do would be better so I could comment directly.

Grahame Grieve (Apr 26 2018 at 21:04):

"A set of resource files
It MAY contain additional content, like example resources or documentation, but this content will be ignored by most package clients and does not follow a standard."

Grahame Grieve (Apr 26 2018 at 21:04):

I think you mean conformance resources in the first sentence given what you say in the second

Grahame Grieve (Apr 26 2018 at 21:05):

why in the manifest example do you have devDependencies which are ignored and not dependencies which are critical?

Martijn Harthoorn (Apr 26 2018 at 21:05):

Yup. just saw that. updated it.

Grahame Grieve (Apr 26 2018 at 21:06):

"It should always have at least one dependency to the core StructureDefinitions package hl7.fhir.StructureDefinitions"... should be just core

Martijn Harthoorn (Apr 26 2018 at 21:07):

done.

Grahame Grieve (Apr 26 2018 at 21:07):

and just realised that if this works well, the other parts of Hl7 will want to use it. so hl7.use.core needs hl7.fhir.us.core or fhir.hl7.us.core

Grahame Grieve (Apr 26 2018 at 21:08):

not sure about the global cache. why do we document that?

Grahame Grieve (Apr 26 2018 at 21:09):

Meta packages: I don't understand why separating out value sets and profiles is a good idea. it just sounds like a problem to me

Martijn Harthoorn (Apr 26 2018 at 21:09):

I see that most package standards seem to have global caches, because the content is the same for each project.

Martijn Harthoorn (Apr 26 2018 at 21:09):

No point in saving it in more than one place.

Grahame Grieve (Apr 26 2018 at 21:09):

doesn't npm cover that?

Martijn Harthoorn (Apr 26 2018 at 21:12):

npm does have a global cache... but that cache might get very big with node (js) modules... And if you want fast / indexed conformance resources, it's probably much faster to have specific .fhir cache over a .npm cache.

Grahame Grieve (Apr 26 2018 at 21:15):

I'm not sure about this. What I know is that many IGs manually include the correct version of the dependency in the version controlled source. the next version of the IG publisher will use npm for dependencies, but so far I haven't figured out how to reduce this necessity

Martijn Harthoorn (Apr 26 2018 at 21:16):

Would including a package.json or package references in the ImplementationGuide resource solve that?

Grahame Grieve (Apr 26 2018 at 21:19):

well, the new IG resource has a dependency structure that maps to the npm logically though it uses canonical URLs (that's going to be pain point). But that doesn't address the need for carrying the actual package around, because the principal use for that is where you're depending on an as yet unpublished version

Martijn Harthoorn (Apr 26 2018 at 21:20):

An unpublished version can still have a name and a version.

Grahame Grieve (Apr 26 2018 at 21:21):

so it can. but authors (a) don't assign the version as they roll over every few minutes and (b) share the package before they commit and publish

Martijn Harthoorn (Apr 26 2018 at 21:21):

i was thinking about version "latest" for that. But that might introduce some problems too.

Martijn Harthoorn (Apr 26 2018 at 21:26):

Btw, Ewout has noted too that the relation between package name and canonical urls do need more thought.

Grahame Grieve (Apr 26 2018 at 21:28):

i think we're going to need automatic conversion at last from canonical -> npm package name. I.m not sure we need the reverse?

Martijn Harthoorn (Apr 26 2018 at 21:35):

https://docs.google.com/document/d/1Skm7Bw_Q4hlfF7rBUay0CIuS9InN91EAyjHX7_QXGcA/edit?usp=sharing

Martijn Harthoorn (Apr 26 2018 at 21:35):

I moved the document here.

Grahame Grieve (Apr 26 2018 at 21:36):

thx

Grahame Grieve (Apr 26 2018 at 23:21):

with regard to package resolution, what would work for me is if we defined 2 special values for version: 'current' (means to get whatever is current from the accepted CI build) and 'local' (means that the only working version is whatever you have locally on your store (whether that's disk or simplifier). If we define those in both IG and packages, that would solve most of the issues I've seen

Grahame Grieve (Apr 26 2018 at 23:23):

what I don't see anywhere is a standard file name for the package. Something like [npm-name].tar.gz

Grahame Grieve (Apr 26 2018 at 23:37):

this would replace the current validation.pack

Grahame Grieve (Apr 26 2018 at 23:40):

and one solution for the package name problem is to put the npm package name in the IG resource.

Grahame Grieve (May 01 2018 at 00:08):

@Martijn Harthoorn I'm about to do a long flight, and I'm going to work on this. When publishing an implementation guide, I'm going to produce a file package.tgz that coontains:
- package/package.json - as described, so far. no doubt this will raise new issues
- package/[type]-[id].json - all the resources in the IG
- package/ig-r4.json - the IG in R4 format
- openapi/[id].schema.json - any generated json schema for the profiles
- openapi/[id].oai.json - any generated open api files for the capability statements
- xml/[id].sch - any generated schematron files

Grahame Grieve (May 01 2018 at 00:08):

I will share them here for people to review

Lloyd McKenzie (May 01 2018 at 01:21):

To confirm, there's no json schema for any 'required' value sets? If there is, then [id].schema.json wouldn't be guaranteed to be unique (though a collision is highly unlikely)

Michel Rutten (May 01 2018 at 06:51):

@Grahame Grieve concerning proposed file names:

package/[type]-[id].json - all the resources in the IG

When saving a new profile, Forge currently proposes a file name of the format "[name].[type].xml" (which the author can override). If desired, I could update the logic to generate file names of the same form "[type]-[id].xml", e.g. "StructureDefinition-MyPatient.xml", to align Forge output with the upcoming NPM tooling.

Lloyd McKenzie (May 01 2018 at 12:59):

That would be useful from just a file-sorting perspective too.

Grahame Grieve (May 01 2018 at 13:29):

json schema is only produced for profiles

Grahame Grieve (May 01 2018 at 13:29):

I don't feel strongly about the naming

Lloyd McKenzie (May 01 2018 at 15:17):

How does JSON Schema handle fixed bindings?

Grahame Grieve (May 02 2018 at 08:04):

enumeration - conceptually like schema

Lloyd McKenzie (May 02 2018 at 12:44):

The real question is "In what file are the enumerations maintained?"

Martijn Harthoorn (May 04 2018 at 08:48):

Note: Not everyone saves an id in conformance resources. It's not mandatory field.

Lloyd McKenzie (May 04 2018 at 12:07):

The comments on Resource.id indicate that the only time it's not present is when submitted for a "create". I can think of a few other times when it could be omitted, but I'd expect tools to enforce its presence in an IG.

Grahame Grieve (May 05 2018 at 03:24):

the IG publisher will always have assigned an id - has to be mandatory as part of an IG

Grahame Grieve (May 05 2018 at 03:24):

the IG publisher doesn't mind if inputs don't have ids

Grahame Grieve (May 05 2018 at 07:28):

ok, here's an example package for us-core:

Grahame Grieve (May 05 2018 at 07:29):

package.tgz

Grahame Grieve (May 05 2018 at 07:29):

comments welcome

Grahame Grieve (May 07 2018 at 08:16):

so the biggest thing that is bugging me about NPM is the difference between npm package ids and canonical urls. Package ids are sort of globally unique, but when presented with a package id, how do you resolve it to an actual package? if it's in the cache, sure, but if it's not...? I don't understand how to do that..

Grahame Grieve (May 07 2018 at 08:18):

so it seems to me that you need something like a packageid.ini in the user fhir package cache that maps from a canonical URL to a package id or vice versa, and that allows a user to specify a local override for a package id (to support development)

Grahame Grieve (May 07 2018 at 08:18):

or have I missed something?

Grahame Grieve (May 07 2018 at 08:18):

@Martijn Harthoorn

Martijn Harthoorn (May 07 2018 at 09:39):

A package server, like registry.npmjs.org can resolve that. We will bring a poc for a FHIR package server to Kohln. But I agree that it would be good if there is a rule / logical relation between a package id and a canonical url.

Grahame Grieve (May 07 2018 at 12:34):

there won't be a rule. I've been playing around with that and it isn't really practical.

Grahame Grieve (May 07 2018 at 12:35):

so I think that we'll have to have a redirect file. but as for a poC for a package server - why woldn't we just use registry.npmjs.org?

Grahame Grieve (May 07 2018 at 12:38):

so the IG publisher is going to migrate to the package based infrastructure by the end of the Koln meeting if we can figure out package resolution.

Grahame Grieve (May 07 2018 at 12:39):

it's producing packages now, following the spec. I'll back generate packages and post them for all the existing published IGs, and then switch to doing dependency fetching using the npm infrastructure

Grahame Grieve (May 07 2018 at 13:09):

btw, @Martijn Harthoorn you haven't commented on my example package...

Grahame Grieve (May 07 2018 at 13:09):

one last thing - the package cache does not contain tarballs? they are extracted?

Martijn Harthoorn (May 07 2018 at 14:59):

Yes. In my poc i extracted them.

Martijn Harthoorn (May 07 2018 at 14:59):

And npm extracts them too.

Martijn Harthoorn (May 07 2018 at 15:07):

btw, @Martijn Harthoorn you haven't commented on my example package...

Looks good. My client tool can install it correctly.

Grahame Grieve (May 07 2018 at 15:33):

great. ok, we go with that for now

Grahame Grieve (May 07 2018 at 15:33):

thx

Grahame Grieve (May 07 2018 at 19:32):

so I asked above: "as for a poC for a package server - why woldn't we just use registry.npmjs.org?".. and the answer appears to be... because of canonical --> npm package id. For which we should use registry.fhir.org, no? The only problem is, it doesn't support the latest IG resource on which I am building all this...

Grahame Grieve (May 07 2018 at 21:16):

ok I have uploaded package.tgz files for all the FHIR base specifications

Grahame Grieve (May 08 2018 at 06:44):

the IGPublisher will support 2 special version values for package versions:
- current - this refers to the most recent build performed by the auto-ig-publisher
- dev - this refers to your local build, or current if no local build exists

These are only valid when developing implementation guides - neither value of version is acceptable when actually publishing

Grahame Grieve (May 08 2018 at 06:47):

@Martijn Harthoorn - both current and dev will be in the global package cache. current will have a timestamp on it - in the root package folder, which should be checked against the auto-builder json registry before using (unless offline). The dev package is written at the conclusion of a successful local build. we'll have to consider how to extend the support for current to simplifier.

Eric Haas (May 08 2018 at 15:14):

Bottom line for editors: When will these be implemented and we have to switch over?

Grahame Grieve (May 08 2018 at 20:08):

you have to 'switch over' now - in that some optional things are now mandatory

Grahame Grieve (May 08 2018 at 20:08):

otherwise, you don't need to do anything to switch over.

Grahame Grieve (May 08 2018 at 20:09):

actually, you might - the version of your dependencies is going to start mattering.

Grahame Grieve (May 08 2018 at 20:10):

@Martijn Harthoorn another issue: there's a 99 character limit on file names. If the resource name is >22 characters long (and we have some) this means that a filename can get longer than that (8+64+5 = 77)

Eric Haas (May 09 2018 at 00:20):

does that mean as soon as I start using the latest ig-pub... everything breaks including my preprocessing?

Grahame Grieve (May 09 2018 at 05:51):

the NPM package file has nothing to do with pre-processing.

Grahame Grieve (May 09 2018 at 05:52):

what you will have to do is version all your dependencies. Which the build tool has not been enforcing, but will when I next commit.

Grahame Grieve (May 09 2018 at 05:53):

as far as I know, nothing else should break... but it depends how far behind you are. You should always use the latest version. You might think you're avoiding pain but you're just kicking it down the road... and things are going to much more stable unless we start making breaking changes to the conformance resources again - in which case you really need to keep up because I'll be writing one off converters....

Grahame Grieve (May 09 2018 at 06:07):

I have uploaded NPM package files for all IGs that are posted to hl7.org or fhir.org for all supported versions. (1.0.2, 1.4.0, 3.01, and 3.2.0+).

Eric Haas (May 09 2018 at 15:04):

what about the CI builds? can we still use a local source file?

Lloyd McKenzie (May 09 2018 at 15:15):

We will definitely need to be able to run the build process totally offline using locally cached files.

Grahame Grieve (May 09 2018 at 19:54):

I don't understand the question, Eric.

Grahame Grieve (May 09 2018 at 19:55):

@Martijn Harthoorn I really hate gzipped tarballs. What a stinking hard format to read... I've spent 4 hours on it so far, and I'm getting nowhere

Michel Rutten (May 10 2018 at 09:53):

@Grahame Grieve I think Martijn uses libraries to deal with gzip/tar, don't think he implemented custom compression logic.

Grahame Grieve (May 10 2018 at 16:07):

I was trying to do the same. Only the libraries for gzip didn't work (i needed a secret undocumented setting, and my google work was deficient in finding it). And there was only one choice for tarball at all, but it worked great.

Grahame Grieve (May 10 2018 at 16:07):

@Martijn Harthoorn ok let's talk about a package server....

Grahame Grieve (May 10 2018 at 16:08):

I know that I need 4 operations:

• list me the packages for a given (partial) package id
• list me packages for a canonical id (should all be the same package, different versions)
• return me a package
• return me a package and all it's dependencies

Grahame Grieve (May 10 2018 at 20:48):

Another issue: I want to be able to use a global package cache from system services too, where there's no user context

Grahame Grieve (May 11 2018 at 07:53):

system package cache (as opposed to user cache) will be at c:\programdata\.fhir or /var/lib/.fhir

Grahame Grieve (May 11 2018 at 07:53):

I will be maintaining files like these now:

Grahame Grieve (May 11 2018 at 07:53):

http://hl7.org/fhir/us/core/package-list.json

Grahame Grieve (May 11 2018 at 07:53):

there's also http://hl7.org/fhir/package-list.json

Grahame Grieve (May 13 2018 at 04:49):

http://wiki.hl7.org/index.php?title=FHIR_NPM_Package_Spec

Grahame Grieve (May 13 2018 at 05:06):

and http://wiki.hl7.org/index.php?title=FHIR_Package_Cache

Grahame Grieve (May 13 2018 at 05:07):

I'm not sure whether the first belongs in the spec? I think not. I'll be asking FHIR-I to adopt both of these as formal policy for FHIR Tooling authors

Lloyd McKenzie (May 13 2018 at 06:03):

My leaning would be to put the first in the spec. It's going to be pretty essential to tool users/developers

Lloyd McKenzie (May 13 2018 at 06:03):

(minor grammar edits to both)

Martijn Harthoorn (May 14 2018 at 09:06):

I know that I need 4 operations:

Martijn Harthoorn (May 14 2018 at 09:07):

• list me the packages for a given (partial) package id
• list me packages for a canonical id (should all be the same package, different versions)
• return me a package
• return me a package and all it's dependencies

Martijn Harthoorn (May 14 2018 at 09:07):

The first and third are covered by the npm spec.

Martijn Harthoorn (May 14 2018 at 09:07):

The last one is a feature addressed by the client in most package resolving standards.

Martijn Harthoorn (May 14 2018 at 09:08):

The main question that we have to answer: do we want to
A - stay within the npm spec,
B - expand on the npm spec,
or C - let it go and define our own.

Grahame Grieve (May 14 2018 at 09:09):

is there an npm field to use for canonical URL?

Grahame Grieve (May 14 2018 at 09:09):

if there is, I'd rather use it. If there isn't, then we have to extend

Martijn Harthoorn (May 14 2018 at 09:13):

As far as I know the whole concept does not exist in npm.

Martijn Harthoorn (May 15 2018 at 12:16):

"list me packages for a canonical id (should all be the same package, different versions)"
1. What is the use case here?

Martijn Harthoorn (May 15 2018 at 12:16):

2. I think it is possible that multiple packages contain the same conformance resource.

Martijn Harthoorn (May 15 2018 at 12:17):

(not just different versions)

Grahame Grieve (May 15 2018 at 20:43):

canonical URL of the IG itself, not of any of the resources in it

Grahame Grieve (May 15 2018 at 20:44):

use case - user knows the package id ,but not what versions are available

Bryn Rhodes (Jun 07 2018 at 20:51):

Updating the QI-Core IG to the latest publisher, the documentation for NPM says the NPM name is assigned by the product director for HL7 IGs. Has one already been assigned for QI-Core?

Grahame Grieve (Jun 07 2018 at 20:54):

it will be hl7.fhir.us.qicore

Bryn Rhodes (Jun 07 2018 at 20:54):

Following the pattern in US core, it would be: hl7.fhir.us.qicore

Bryn Rhodes (Jun 07 2018 at 20:55):

Ah yes. same time.

Bryn Rhodes (Jun 07 2018 at 21:02):

This new publisher is the cat's meow: NPM support, license declaration, nice work!

Bryn Rhodes (Jun 07 2018 at 21:03):

The docs are really good too.

Grahame Grieve (Jun 07 2018 at 21:04):

thanks. I'm glad someone is happy

Oliver Egger (Jun 07 2018 at 21:07):

i'am very happy to, much cleaner separation, possibility to specify the ig for validation, thanks a lot!

Eric Haas (Jun 08 2018 at 14:34):

I admit I don't understand the whole package thing. I'm doing the same thing in generating igs as before. So I am not personally seeing any differences in creating a npm file. What are the benefits again, who is the big beneficiary, and am I ignoring something I should be taking advantage of as an ig author?

Bryn Rhodes (Jun 08 2018 at 18:18):

Referencing the IG can be done with standard NPM package managers now, makes it easier to plug in to existing tooling.

Grahame Grieve (Jun 08 2018 at 20:18):

the big problem I had was how to manage references. There was a location property on dependency in the json file, but everyone had to have the same literal path for that. Now, the package infrastructure can auto-resolve local references. that's the primary benefit for IT publishers, and more than justified the trivial cost of having to define npm-name and license.

Grahame Grieve (Jun 08 2018 at 20:18):

otherwise there is many downstream benefits for tooling

Eric Haas (Jun 08 2018 at 20:27):

ok got it

Sarah Gaunt (Jun 11 2018 at 08:00):

@Grahame Grieve for the Case Reporting IG, I have set "npm-name": "hl7.fhir.us.ecr" - is this correct?

Grahame Grieve (Jun 11 2018 at 08:03):

no. it's hl7.fhir.uv.ecr (you were close though)

Sarah Gaunt (Jun 11 2018 at 08:03):

Ok, isn't uv universal realm? This is US realm...

Grahame Grieve (Jun 11 2018 at 08:13):

its not US realm. So I was assured when the project proposal went through. see http://hl7.org/fhir/uv/ecr/

Sarah Gaunt (Jun 11 2018 at 08:13):

Ha! News to me... Will look into it!
Pretty sure it wouldn't be of much use to us here, that's for sure...

Grahame Grieve (Jun 11 2018 at 08:15):

that's what I thought...

Sarah Gaunt (Jun 11 2018 at 08:15):

Maybe the FHIR version is supposed to be uv. Though it's pretty much a copy of the CDA, which is definitely US Realm. Looking for PSS now.

Sarah Gaunt (Jun 11 2018 at 08:17):

http://www.hl7.org/special/Committees/projman/searchableProjectIndex.cfm?action=edit&ProjectNumber=1366

Sarah Gaunt (Jun 11 2018 at 08:18):

It's def US realm.

Sarah Gaunt (Jun 11 2018 at 08:18):

Is it something we need to fix now or can it wait until later?

Grahame Grieve (Jun 11 2018 at 08:20):

I'm not sure.

Sarah Gaunt (Jun 11 2018 at 08:22):

Maybe later - more concerned with getting the CI build working again right now. Clearly I also have canonical base path wrong in ig.json (us vs uv).

Grahame Grieve (Jun 11 2018 at 08:23):

it was balloted as UV - see http://www.hl7.org/documentcenter/public_temp_4B4D4D8F-1C23-BA17-0CB0A5D31878A17B/ballots/2018JAN/Announcements/Announcement%20of%20Ballot%20Openings%20for%202018%20January%20Ballot%20Cycle.pdf

Sarah Gaunt (Jun 11 2018 at 08:23):

Wow, crazy. I'll talk to the powers that be and get back to you tomorrow...

Sarah Gaunt (Jun 11 2018 at 08:30):

That was the problem with the CI build - the US vs UV issue... It's running now. Not sure how it was running last week...

Sarah Gaunt (Jun 11 2018 at 08:32):

Except now it's failing with stuff like this:
base/ resource url mismatch: http://hl7.org/fhir/uv/ecr vs http://fhir.hl7.org/us/ecr/StructureDefinition/extension-trigger
I will leave it for now and find out what the project is really supposed to be UV or US realm - otherwise I'll be changing all the US to UV and will likely have to change them back later.

Sarah Gaunt (Jun 11 2018 at 19:34):

@Grahame Grieve So eCR is definitely supposed to be US Realm NOT UV. John Loonsk thinks the confusion may have come about because originally there was an idea that we might need to create a new public health resource - which obviously wouldn't have been US Realm? It should really have been balloted as US realm...

So, can you change the npm name to US and http://hl7.org/fhir/uv/ecr/ to reflect US Realm also?

Grahame Grieve (Jun 11 2018 at 22:57):

I don't think I can fix this.... will have to talk to FMG about it

Sarah Gaunt (Jun 12 2018 at 01:20):

Ok.
Is there anything I need to do? Or is it something you need to take care of?

Sarah Gaunt (Jun 12 2018 at 01:22):

The NIB: http://www.hl7.org/special/Committees/tsc/ballotmanagement/ViewPastNIB.cfm?ballot_pool_id=1151
states that it's US specific...

Martijn Harthoorn (Jun 14 2018 at 09:55):

Is any one interested in a connectathon track for FHIR packages in Baltimore (September)?

Michel Rutten (Jun 14 2018 at 09:56):

Quoting Grahame:

there's two parts to test: local cache interoperability and package resolution consistency

Grahame Grieve (Jun 14 2018 at 09:57):

and yes i have to be interested though I'll be under immense pressure. we can do it next weje in Boston?

Sarah Gaunt (Jun 18 2018 at 20:43):

@Grahame Grieve (@Rick Geimer) Grahame - is this something you can take to FMG this week? This issue is holding me up as the IG publisher won't build until the US vs UV mismatches are sorted (and I really don't want to go and rename everything to UV!)

Grahame Grieve (Aug 23 2018 at 05:59):

back to this: @Martijn Harthoorn the package spec makes no rules around the possible values for the version string, and I just got bitten by this. We need to at least say: shall not contain a space or characters /\:;,?

Martijn Harthoorn (Aug 23 2018 at 20:48):

Our spec currently says it should adhere to Semver and "Version strings SHALL contain only letters, numbers, and the characters ".", "_", and "-"

Martijn Harthoorn (Aug 23 2018 at 20:49):

Or do you mean something else?

Martijn Harthoorn (Aug 23 2018 at 20:49):

http://wiki.hl7.org/index.php?title=FHIR_NPM_Package_Spec

Grahame Grieve (Aug 23 2018 at 21:11):

I drafted the

Version strings SHALL contain only letters, numbers, and the characters ".", "_", and "-"

slightly before I wrote what I wrote here

Grahame Grieve (Aug 23 2018 at 21:12):

I think we should update to include the other characters but I was looking for confirmation here on the details

Martijn Harthoorn (Aug 24 2018 at 04:56):

Ah. That explains. The initial proposal had a MUST for semver. I think there is still a strong case for that, since you tooling can use it to calculate compatibility. If people choose not to follow semver, it's much less certain whether a tool can still do that.

Martijn Harthoorn (Aug 24 2018 at 04:58):

Semver says you can only have numbers for major, minor and patch. Except for the prerelease which can contain alphanumerics and a dash. (and a more optionally + and an alpha-identifier for a build)

Martijn Harthoorn (Aug 24 2018 at 05:00):

For people, organizations that can't or won't follow semver - or whatever form of computability we choose, I don't see why we care what is in the version string at all.

Martijn Harthoorn (Aug 24 2018 at 05:01):

But personally I don't mind setting up some rules for it, like dot, underscore or dash and not spaces.

Grahame Grieve (Aug 24 2018 at 05:07):

well, because the cache specification is unimplementable if characters that are not valid file names on all platforms are allowed in the version

Last updated: Apr 12 2022 at 19:14 UTC