FHIR Chat · Package rework · conformance

Stream: conformance

Topic: Package rework

Grahame Grieve (Jun 11 2019 at 18:43):

Follow up to last night @Martijn Harthoorn @Michel Rutten removing narratives from r4 package reduces size from 16615 kb to 13283 kb

Grahame Grieve (Jun 11 2019 at 18:50):

removing exmples and data element definitions reduces the package size to 11639 kb, and the overall resource count from 5535 to 4603 files

Michel Rutten (Jun 11 2019 at 18:51):

That is a significant decrease in size

Grahame Grieve (Jun 11 2019 at 18:54):

so what does that imply? The obvious thing to do is

hl7.fhir.core.r4
hl7.fhir.core.r4.examples

Though that means that .examples is not a subset of the hl7.fhir.core.r4... which isn't what the current spec says.

Grahame Grieve (Jun 11 2019 at 20:23):

do we need to do anything more

Jim Steel (Jun 11 2019 at 20:23):

Has there been a check that all of the terminology resources are still used/necessary?

Jim Steel (Jun 11 2019 at 20:23):

(Not basing this on knowledge, just a hunch)

Grahame Grieve (Jun 11 2019 at 20:26):

how would you decide what 'not necessary' means? Forge doesn't use any of them... the rest of us might use any of them

Jim Steel (Jun 11 2019 at 20:26):

Yeah, its a good question. The ones I worry about particularly are the ones the spec used to use but doesn't any longer

Grahame Grieve (Jun 11 2019 at 20:27):

which ones?

Jim Steel (Jun 11 2019 at 20:28):

If I knew that it'd already be a ticket :)

Grahame Grieve (Jun 18 2019 at 06:03):

@Martijn Harthoorn can we close this up. Open issues:
- comments about splitting core package up
- agreeing format for the cache file

Martijn Harthoorn (Jun 20 2019 at 13:17):

@Grahame Grieve , although I believe from a technical perspective I see some good reasons to split up the core, without circular references. I also recognize that it makes it too hard to explain for non technical users. So my guess is that we should just give it a go. Your numbers look a lot better to work with.

Martijn Harthoorn (Jun 20 2019 at 13:18):

Can you give a summary for the cache file once more? Or do you have a link?

Grahame Grieve (Jun 20 2019 at 19:18):

give a summary for the cache file

not sure what you mean?

Martijn Harthoorn (Jun 21 2019 at 14:44):

I remember we discussed the format, and that in itself there were no issues from our side, but I can't remember the specifics of the format, that you showed.

Grahame Grieve (Jun 21 2019 at 20:11):

ok. I want to record the date that the package was installed into the cache, and also record a canonical URL list - a mapping from canonical URL to resource id

Grahame Grieve (Jun 21 2019 at 20:16):

I currently create a file cache.ini that looks like:

[Package]
install = 14/04/2019 7:47:35 AM

[Profiles]
http://hl7.org/fhir/StructureDefinition/sdc-valueset = ValueSet

Grahame Grieve (Jun 21 2019 at 20:17):

you said that a cache file like this was of interest, and it would be good for us to agree about this. You also said that you might want a different format...

Grahame Grieve (Jun 23 2019 at 21:16):

also. this is the right arrangement:

• hl7.fhir.r4.core
• hl7.fhir.r4.examples

Martijn Harthoorn (Jun 24 2019 at 08:17):

Yes.

Martijn Harthoorn (Jun 24 2019 at 08:19):

Regarding the cache.ini, I will discuss it today with @Ewout Kramer and @Michel Rutten , since they have a use case for indexes on other fields. So I think the best format I would probably prefer a .json format, since it's more flexible to add other index information.

Grahame Grieve (Jun 24 2019 at 10:39):

well, ok... but what would it look like?

Martijn Harthoorn (Jun 24 2019 at 15:11):

I had a short discussion about it with Ewout and we come to the conclusion that there are no stringent needs, other than indexing the canonicals to files for now.

Martijn Harthoorn (Jun 24 2019 at 15:11):

The format we use now is this json format:

Martijn Harthoorn (Jun 24 2019 at 15:11):

{
  "canonicals": {
    "http://hl7.org/fhir/StructureDefinition/actualgroup": "StructureDefinition-actualgroup.json",
    "http://hl7.org/fhir/StructureDefinition/bmi": "StructureDefinition-bmi.json",
    "http://hl7.org/fhir/StructureDefinition/bodyheight": "StructureDefinition-bodyheight.json",
    "http://hl7.org/fhir/StructureDefinition/bodytemp": "StructureDefinition-bodytemp.json",
    "http://hl7.org/fhir/StructureDefinition/bodyweight": "StructureDefinition-bodyweight.json",
    "http://hl7.org/fhir/StructureDefinition/bp": "StructureDefinition-bp.json",
    ...
  }
}

Grahame Grieve (Jun 24 2019 at 20:46):

ok can we add date?

Grahame Grieve (Jun 24 2019 at 20:47):

{
  "date" : "2019-06-26T06:47:00+10:00",
  "canonicals": {
    "http://hl7.org/fhir/StructureDefinition/actualgroup": "StructureDefinition-actualgroup.json",
    ...
  }
}

Grahame Grieve (Jun 24 2019 at 20:47):

sole use is display to a user that X has been in the package cache for d duration

Grahame Grieve (Jun 25 2019 at 08:35):

.. and when we can we start talking about a server API? I have a place in the tools for posting an external package to registry.fhir.org now....

Martijn Harthoorn (Jun 25 2019 at 08:45):

Yes. I am fine with adding the date.

Martijn Harthoorn (Jun 25 2019 at 08:46):

I 'll start doing that on our side too.

Martijn Harthoorn (Jun 25 2019 at 08:54):

Shall we call it "_index.json"?

Vadim Peretokin (Jun 25 2019 at 10:06):

I'd prefer .index.json if you want to indicate it's an infrastructure file. Underscores are a C# notion, not used in file naming

Grahame Grieve (Jun 25 2019 at 10:55):

that's just to make it hard to create the file manually on windows ;-)

Martijn Harthoorn (Jun 25 2019 at 13:31):

Windows is doing much better in that respect, these days.

Martijn Harthoorn (Jun 25 2019 at 13:32):

I am good with either.

Grahame Grieve (Jun 25 2019 at 13:32):

explorer still won't let you create them manually

Grahame Grieve (Jun 25 2019 at 13:33):

but I'm ok with .index.json

Martijn Harthoorn (Jun 25 2019 at 13:34):

Ok. Let's do that. The people that want to create a canonical index file manually, are probably sufficiently equiped to navigate around explorer. :)

Grahame Grieve (Jun 25 2019 at 13:34):

nobody should create them manally.

Martijn Harthoorn (Jun 25 2019 at 13:34):

exactly.

Grahame Grieve (Jun 25 2019 at 13:36):

ok time to update this page - http://wiki.hl7.org/index.php?title=FHIR_Package_Cache - do you want to do that, or me?

Grahame Grieve (Jun 25 2019 at 13:36):

and can we also agree about packages.ini as wel

Martijn Harthoorn (Jun 25 2019 at 13:37):

What is packages.ini?

Martijn Harthoorn (Jun 25 2019 at 13:45):

"cache: version = 2 - indicates that the contents of the package cache conform to this specification" -- I am not sure what that means.

Martijn Harthoorn (Jun 25 2019 at 13:45):

Is that the FHIR specification? Or the package cache specification?

Grahame Grieve (Jun 25 2019 at 13:46):

package cache specification

Martijn Harthoorn (Jun 25 2019 at 13:48):

From a puristic perspecfive I understand the usecase. But I don't think I have the time to write tooling that understands more than version of the global package cache.

Grahame Grieve (Jun 25 2019 at 13:48):

oh no neither do I. I just wipe the cache when the version is wrong and pretend it's empty. It's a safety thing

Martijn Harthoorn (Jun 25 2019 at 13:49):

:)

Grahame Grieve (Jun 26 2019 at 09:45):

ok draft new packages here:

Grahame Grieve (Jun 26 2019 at 09:46):

https://github.com/FHIR/packages/tree/master/core

Grahame Grieve (Jun 26 2019 at 09:47):

please review, particularly the following:
- the values in the various package.json files
- especially version
- I don't know what to do with the semi-examples for code system etc - sometimes they are purely example, sometimes they are more

Grahame Grieve (Jun 26 2019 at 09:48):

I'm not sure whether it's useful to have a hl7.fhir.rX which includes... core + examples. I guess. expansions and elements are both special purpose packages, and expansions has duplicate urls

Grahame Grieve (Jun 27 2019 at 13:08):

updated. core is parallel to examples;; there's no valid combination of the packages. they each haev their own purpose.

Grahame Grieve (Jun 27 2019 at 13:08):

version is still an open topic

Martijn Harthoorn (Jun 28 2019 at 08:35):

You mean the relation between the FHIR version and the package version, right?

Martijn Harthoorn (Jun 28 2019 at 09:06):

There are some packages that refer to hl7.fhir.core instead of hl7.fhir.core.r2 / r3 / r4.
For example:
https://github.com/FHIR/packages/blob/master/fhir.test.data/4.0.0/package/package.json

Grahame Grieve (Jun 29 2019 at 03:22):

I’ll have to rework all of them. I haven’t done that yet. My question is what the package version should be for the core packages

Grahame Grieve (Jul 05 2019 at 00:40):

ping...

Martijn Harthoorn (Jul 05 2019 at 11:04):

Good question. :)

Martijn Harthoorn (Jul 05 2019 at 11:23):

I had a brief discussion with Ewout and his recollection is that this was decided in an evening session at Montreal, between you Ewout, Josh, and others, with the following conclusion:
- MAJOR/MINOR follow the HL7 FHIR standard version
- PATCH is for updates to the package itself that are not caused by the standards process.

Grahame Grieve (Jul 05 2019 at 11:24):

well, that's what we're doing with the IG packages. And that's also what we said we'd do with the core packages too, but it isn't clear to me that it's still the right decision after we decide to include the major version in the package id too

Grahame Grieve (Jul 05 2019 at 11:24):

but no one has offered any alternative...

Martijn Harthoorn (Jul 05 2019 at 11:25):

Yeah. And I think it will be confusing to some people when you release r3. version 1.0.0 :)

Martijn Harthoorn (Jul 05 2019 at 11:25):

And everything besides 3 (FHIR version) and 1.0 - the normal start is even crazier.

Grahame Grieve (Jul 05 2019 at 11:35):

ok that means that the packages found at https://github.com/FHIR/packages are all good to go (except for old_core which are no longer relevant and I'll delete once we pull the trigger on this change...)

Martijn Harthoorn (Jul 05 2019 at 11:45):

So that's the stuff under hl7.fhir.rX, right?
https://github.com/FHIR/packages/tree/master/hl7.fhir.rX

Martijn Harthoorn (Jul 05 2019 at 14:48):

I uploaded it to our staging area, and was able to create and consume the package.

Martijn Harthoorn (Jul 05 2019 at 14:48):

(I used hl7.fhir.r3.core)

Martijn Harthoorn (Jul 05 2019 at 14:48):

I could validate resources with it in Torinox.

Martijn Harthoorn (Jul 05 2019 at 14:48):

So I think we are ready too.

Grahame Grieve (Jul 05 2019 at 20:52):

good

Grahame Grieve (Jul 05 2019 at 21:09):

ok. http://wiki.hl7.org/index.php?title=FHIR_Package_Cache updated. please check

Grahame Grieve (Jul 05 2019 at 21:11):

also, we still need to work on a package API....

Grahame Grieve (Jul 06 2019 at 22:22):

ah no - at DevDays @nicola (RIO/SS) proposed to use subdirectories for types, so that ValueSet XX would be package/ValueSet/XX.json instead of package/ValueSet-XX.json. We sort of informally agreed to that change, but it's pretty significant , so let's reconfirm here in an audited way

Martijn Harthoorn (Jul 08 2019 at 09:43):

There are two parts to this:
1. Should we allow subfolders in the "package" folder? -- I think the answer is yes. No problem.
2. Should we make it mandatory to have resource type in the corresponding sub folder with the type name?
It's easier for indexing and searching, but I am afraid we are pinning ourselves down too much. And are we then going to create 100 sub folders with one or two examples in each? Maybe make it a 'SHOULD'...

Grahame Grieve (Jul 08 2019 at 19:52):

I don't want to make it a choice. One way or the other. It is a problem to have 100 sub-folders?

Martijn Harthoorn (Jul 09 2019 at 08:37):

There is no added value in making it mandatory. Only burden and lack of choice.

Martijn Harthoorn (Jul 09 2019 at 08:38):

I am fine with (optionally) allowing sub folders - npm already has that.

Grahame Grieve (Jul 10 2019 at 21:30):

why would we require package consumers to check both package and package subfolders?

Martijn Harthoorn (Jul 11 2019 at 11:27):

Are saying you don't want to allow sub folders?

Grahame Grieve (Jul 11 2019 at 12:07):

well, I don't see the point of allowing resources to be either in the package folder, or a sub-folder of that. I don't care whether there is sub-folders, only whether tools are expected to find resources in sub-folders or not.

Grahame Grieve (Jul 11 2019 at 12:07):

@nicola (RIO/SS) suggested that it would make loading faster, but of course if it can be either way, it would make loading slower

nicola (RIO/SS) (Jul 11 2019 at 12:09):

this is not about speed, this is about aesthetic

nicola (RIO/SS) (Jul 11 2019 at 12:12):

i think, developers will like to have all search params in a separate folder

nicola (RIO/SS) (Jul 11 2019 at 12:13):

if you are talking about examples, i would put them under /examples folder ;)

Grahame Grieve (Jul 11 2019 at 12:19):

well, I don't care about that so much for tooling purposes...

Grahame Grieve (Jul 11 2019 at 12:19):

but I stick to my position: this should not be a choice: we should say, one way or the other

Grahame Grieve (Jul 11 2019 at 12:19):

I don't really care which way personally

Grahame Grieve (Jul 11 2019 at 12:19):

hmm. that one is tricky. We don't actually have a clear differentiation between 'examples' and not-examples.

Grahame Grieve (Jul 11 2019 at 12:19):

I have split into 2 packages - 'examples' and 'core'. But they actually overlap - examples contains everything in core, but with narrative. Core has no narrative, and just conformance and terminology resources - whether labelled examples or not, since some of the examples are real things anyway.

Martijn Harthoorn (Jul 16 2019 at 14:04):

• I am ok with not allowing subfolders
• I am ok with allowing sub folders
• I feed bad about forcing subfolders.

Grahame Grieve (Jul 24 2019 at 21:59):

we need to close this out. No subfolders. If anyone objects, I guess I'll refer it to FHIR-I

Martijn Harthoorn (Jul 26 2019 at 14:12):

Ok.

Last updated: Apr 12 2022 at 19:14 UTC