Stream: implementers
Topic: StructureDefinitions on github
Patrik Sundberg (Sep 12 2018 at 01:59):
Is there any plan to publish the structuredefinitions for dstu2, stu3, r4 on github directly? That way packages that need to use these artifacts can get them from there instead of having to clone them, like we do here: https://github.com/google/fhir/tree/master/testdata/stu3/structure_definitions
i'm happy to help set it up.
Grahame Grieve (Sep 12 2018 at 02:00):
we're using npm to distribute these
Grahame Grieve (Sep 12 2018 at 02:01):
there's npm packages at http://hl7.org/fhir/STU3/package.tgz etc
Patrik Sundberg (Sep 12 2018 at 03:33):
Right. One thing github gives you is the ability to reference a specific repo version. That lets a user update the pointer to the spec in their repo in the same change as any downstream generated code that would be modified as a result of the spec changing. Without this functionality, changing the fhir package may break downstream builds. I'm not that familiar with npm but i don't believe it provides that functionality.
Grahame Grieve (Sep 12 2018 at 03:40):
well, I thought that was the point
Grahame Grieve (Sep 12 2018 at 03:41):
I mean, of the npm package thing
Grahame Grieve (Sep 12 2018 at 03:41):
perhaps I don't understand the context, but you reference the package you want...
Lloyd McKenzie (Sep 12 2018 at 13:52):
The packages are version-specific
Grahame Grieve (Sep 15 2018 at 23:03):
except for current
Grahame Grieve (Sep 15 2018 at 23:04):
I don't know if this is relevant, but I've moved some of my server support to https://github.com/FHIR/packages - this is the various resources that are useful for servers for testing but not suitable for the spec itself
Grahame Grieve (Sep 15 2018 at 23:04):
some is sourced from MIHIN (thanks)
Grahame Grieve (Sep 15 2018 at 23:04):
contributions are welcome.
Grahame Grieve (Sep 15 2018 at 23:04):
I suppose we could put the structure definitions up there too.... I'm just not clear on the value proposition right now
Patrik Sundberg (Sep 16 2018 at 04:59):
from what i can tell you were referring to this npm package: https://www.npmjs.com/package/fhir-schemas
which seems to be generated from source here: https://github.com/clinical-meteor/fhir-schemas
that package doesn't contain the structure definitions in json format, the schemas are in json-schema. is there another npm package i should be looking at?
the testdata you've posted does seem relevant to me, we'll have a look at it. also, i personally like github a lot better, because it's easier to integrate with as a dependency. putting the structure definitions in your new package would help us out.
Patrik Sundberg (Sep 16 2018 at 07:20):
just found https://chat.fhir.org/#narrow/stream/99-IG-creation/subject/NPM.20Package.20File
i think i was missing a lot of context. So, you're building npm packages but they are not published on the npmjs site, instead you list them all in http://hl7.org/fhir/package-list.json and similar. The actual data is in $path/package.tgz, and these files have a lot more data in them than the package i found on npmjs, including (i think) the stuff i want. Overall this setup looks quite nice. Now I just have to see if I can integrate it with our build tools.
I'm assuming packages are immutable once published? If so, can you also add checksums to the package list?
Lloyd McKenzie (Sep 16 2018 at 15:57):
They haven't been immutable in practice as we've had to fix various errors in them and/or add content to older ones needed by newer tools.
Patrik Sundberg (Sep 16 2018 at 18:12):
then we're back to why i would have liked to see these packages on github. various downstream artifacts are generated from the structure definitions; if the structure definitions change, downstream artifacts will also change, and suddenly the build will break. or worse, the build won't break, but depending on when you downloaded the dependencies, it will produce different results. what you're describing sounds more like release branches, which occasionally get updated, ie hl7.fhir.core.stu3 v1.0.0 to v1.0.1 when fixing errors or v1.1.0 when adding content needed by newer tools. or am i off base here?
Lloyd McKenzie (Sep 16 2018 at 21:55):
The formal models don't change. What was legal as a Patient in STU2 is still the same. However, for example, the FHIRPath language has changed and become the primary mechanism for expressing constraints. To work with the tools, the rules as they were for STU2 need to be expressed in a way the tools can support. Also, in some cases, we had models that were invalid and didn't notice because the tooling at that point couldn't detect the issue. So we updated the formal representation to reflect what the ballot actually said in the user presentation.
Patrik Sundberg (Sep 16 2018 at 22:38):
this all makes sense, but the problem still remains: if i have unittests that for some obscure reason depend on some bug in the models, these tests will break when you fix the bug and push an updated package. imho a better way of handling this situation is to mark the existing package invalid or deprecated, push a new one with a new version number, and let downstream consumers atomically switch out the dependency and fix the unittests.
the situation is particularly bad for deeper dependencies. let's say i'm writing a package which has versions 1.0.0, 1.1.0, and 2.0.0, and other users depend on my package. Some users are happy with 1.x, but all new development is on 2.x. When the underlying fhir model package changes, all three of these versions may break, and require patches. Now I have to fix all branches of my package, and then all users of my package have to update their dependencies to unbreak their builds. This is exactly the situation package versioning is meant to avoid.
If the package.tgz files were pushed to github, in the same directory structure they exist now on the fhir website, ie
http://hl7.org/fhir/us/core/2018Jan/package.tgz -> https://github.com/FHIR/packages/us/core/2018Jan/package.tgz, github would provide the necessary versioning, and i'd be happy.
Patrik Sundberg (Sep 16 2018 at 23:26):
(i'd probably be even happier if the packages were expanded fully into individual files, since github will automatically .tar.gz the content anyway, and doing it twice seems unnecessary)
Grahame Grieve (Sep 17 2018 at 09:29):
well, there's kind of 2 different issues here.
Grahame Grieve (Sep 17 2018 at 09:33):
We'll have to talk about the versioning policy. We actually haven't made updates the tgz like Lloyd talks about - he's referring to older things. But we might still have to do that. And it wouldn't work well. But we can't version the packages the same way as a primary packge - it's a secondary asset
Patrik Sundberg (Sep 18 2018 at 04:35):
in practice, if you can promise that the updates are "rare" for some definition of rare it might still be enticing to try these packages out now, while the debate for how to handle updates goes on.
Grahame Grieve (Sep 18 2018 at 11:17):
they are very rare for anything older than 3.6.0.
Patrik Sundberg (Sep 18 2018 at 19:27):
fyi, this: https://github.com/google/fhir/commit/d0eb78f70ced5e5c3e0d23eaef13e4cf1918c8d6#diff-25fd6a5ac4c10dd98d574a5b4004f777 is the only real change we had to hack into the definitions to get the protobuf generation working. how would an update like that work in this model?
Grahame Grieve (Sep 18 2018 at 20:10):
I'm not sure what you're asking
Patrik Sundberg (Sep 18 2018 at 22:55):
if we filed a tracker request to make this change, and that request was approved, and it was decided that the official structure definitions should be updated:
- would this process result in a 3.0.2 release?
- would this update be included in a larger set, and that release be 3.0.2?
- would there be an update to the structure definitions, but the version would be kept at 3.0.1 still because the edits were minor?
- is this question moot because these 3.0.1 structure definitions will remain the final stu3 set?
just trying to understand the versioning policy.
Grahame Grieve (Sep 19 2018 at 00:12):
I'm not sure. we might handle it as a technical correction, or not. We might not make the change at all. And then the version policy would follow from that. we might issue a set of informal corrections... I just don't know. All the options are not great. I wish I could offer more clarity
Last updated: Apr 12 2022 at 19:14 UTC
