Stream: tooling
Topic: Supporting resource instance versions
Jose Costa Teixeira (Feb 25 2020 at 18:43):
Not sure where to post this, since it may impact IG tooling , templates, and sushi and other tools-
In order to prepare for eventual support for multiple versions of assets, should we have a file naming convention?
Jose Costa Teixeira (Feb 25 2020 at 18:46):
I.e. if I have a patient id=001, which has versions 0, 1 and 2, I need to store the jsons somewhere in my file system, and I don't want then all to be called patient-001.json.
how about patient-001#0.json, patient-001#1.json, patient-001#2.json ?
Lloyd McKenzie (Feb 25 2020 at 19:00):
Are Windows, Mac and Unix file systems comfortable with '#' in a filename? (It'd be super surprising to me if any were, let alone all of them.)
Jose Costa Teixeira (Feb 25 2020 at 19:01):
windows is
Jose Costa Teixeira (Feb 25 2020 at 19:04):
linux is
Jose Costa Teixeira (Feb 25 2020 at 19:04):
and mac is unix, so I expect it to be as well
Jose Costa Teixeira (Feb 25 2020 at 19:04):
(the fhir package cache uses that notation also to support versions)
Jose Costa Teixeira (Feb 25 2020 at 19:09):
it will be an issue in a url, i think. but i'm not sure that matters - or any other idea?
Grahame Grieve (Feb 25 2020 at 19:42):
# is ok on windows|unix|mac. But the tooling isn't ok with it anywhere
Jose Costa Teixeira (Feb 25 2020 at 20:19):
i understand no tooling supports it yet. But if we have an agreement on this then tooling can evolve to that
Jose Costa Teixeira (Feb 25 2020 at 21:07):
The problem is we often have URLs that point to stuff inside a file - and the syntax for that uses '#'
(@Lloyd McKenzie i presume you meant to post here)
ok I can see an issue if we want to download a file called patient001#001.json
Chris Moesel (Feb 25 2020 at 21:08):
I don't know what OS support is for | -- but using that as a version separator would be consistent w/ canonical and has the advantage of not being mistaken as a URL fragment identifier. The downside is that it's considered an "unsafe" character in URLs, so must be escaped in that context.
Jose Costa Teixeira (Feb 25 2020 at 21:09):
so neither of these work in urls.
Ward Weistra (Feb 26 2020 at 09:58):
@Jose Costa Teixeira Sorry if I miss the backstory, but is there an important use for multiple version of the same resource in one project/folder/ig/package? If these are different versions of the resource corresponding to different versions of a project/ig/package I guess they could just live in different branches or folders?
Jose Costa Teixeira (Feb 26 2020 at 13:56):
Hi. Yes, this is needed for anything that is providing guidance on workflows or dynamic behaviour (whether it is important or not, depends on where you stand).
Example: CarePlan001 has a first version with status Draft, then have an update with different status. At this moment I cnnot create those in an IG, like I would have on a server.
Ward Weistra (Feb 26 2020 at 14:06):
Thanks @Jose Costa Teixeira, so these versions reflect the example instance as it would be in different stages of the workflow. Then they indeed need to be in the same project/ig/package, it's not a | versioning situation.
Jose Costa Teixeira (Feb 26 2020 at 14:16):
the # and the | (I think I prefer the # ) is just the file name. If they are all called CarePlan001, it does not work on a file system
Jose Costa Teixeira (Feb 26 2020 at 14:16):
an alternative:
we could use the / as separator
John Moehrke (Feb 26 2020 at 14:28):
or abandon hope of maintaining human understandable filenames. The goal is laudable, but you will notice in systems like google drive and confluence and others; there comes a point when everything breaks down and GUIDs are used... preferring external (or internal) means to take over
Jose Costa Teixeira (Feb 26 2020 at 14:30):
the problem still applies if we use uuid - at least in the file name if the filename is the same as the identifier
Jose Costa Teixeira (Feb 26 2020 at 14:31):
you can have careteam "a2b8555d-622e-43f2-8412-baf0e22bb050.json" but you need two versions, and those two files cannot be called "a2b8555d-622e-43f2-8412-baf0e22bb050.json"
John Moehrke (Feb 26 2020 at 14:33):
exactly correct. both would be different guids, from the filename perspective there is no relationship. The relationship is only internally as part of the FHIR model, or externally as part of your test scenario
Jose Costa Teixeira (Feb 26 2020 at 14:36):
so as an alternative we could have something like:
| resourceid | version | filename |
| careteam001 | v0 | a2b8555d-622e-43f2-8412-baf0e22bb050.json |
| careteam001 | v1 | 825d4a5c-56c7-4a0e-ade6-a2e6ee28f699.json |
| careteam001 | v2 | 7762306b-c276-4049-a4d1-54abb3a28893.json |
Jose Costa Teixeira (Feb 26 2020 at 14:36):
this?
John Moehrke (Feb 26 2020 at 14:38):
not saying I like it.. Just noting that many web tools try to maintain human readable filenames, but get to a point where they totally breakdown and become completely opaque. So, just setting the stage that we might not be able to invent what many others have tried to invent and failed.
Jose Costa Teixeira (Feb 26 2020 at 14:45):
we do use # so far, it works where we need it to work
Grahame Grieve (Feb 26 2020 at 17:19):
the point is that that the tools already allow you to vary the file name in the input, but are completely driven by resource id internally, and so are unable to manage having 2 different instances of the same thing. This applies to all internal reasoning, and also to to the way that the web output is generated.
Grahame Grieve (Feb 26 2020 at 17:19):
trying to change that is pulling the logical carpet from under everything I've ever done
Lloyd McKenzie (Feb 26 2020 at 18:05):
It would be useful to have multiple versions of the same instance for examples. I believe the tooling can handle that when the content is inside a Bundle. Perhaps we could find a way to drive the publication of multiple instances from within a Bundle?
Jose Costa Teixeira (Mar 04 2020 at 15:18):
Should "resource#v1" be acceptable as an identifier.value?
Lloyd McKenzie (Mar 04 2020 at 17:17):
Identifier.value is a string. So long as it's less than a megabyte, it can contain any combination of UTF-8 characters you like.
Lloyd McKenzie (Mar 04 2020 at 17:18):
Consumers will treat it as a string though - no one will recognize or parse the #v1 as having any meaning and it would be bad practice to require systems to do so.
Jose Costa Teixeira (Mar 04 2020 at 17:45):
when I tried it, the IG Publisher was giving me a "....must be a valid identifier..."
Jose Costa Teixeira (Mar 04 2020 at 17:47):
and a correction: I actually had "resource#1", not "resource#v1"
Jose Costa Teixeira (Mar 04 2020 at 17:48):
I'm just looking for a consistent / sensible way to represent resource versions. If the tools do not support it yet, that is OK.
But if we have one syntax, then the tools can catch up (if the toolsmiths want).
Lloyd McKenzie (Mar 04 2020 at 17:53):
If you're pointing to resources, you ought to be using Reference, not Identifier
Jose Costa Teixeira (Mar 04 2020 at 18:06):
in the ig.xml?
Jose Costa Teixeira (Mar 04 2020 at 18:07):
the issue I had was when I told the IG that the resource was found like this:
<reference>
<reference value="CareTeam/dzop-team-0002#2"/>
</reference>
Lloyd McKenzie (Mar 04 2020 at 19:05):
You said identifier.value. What you're showing now is reference.value. reference.value is appropriate. Right now, the IGPublisher doesn't support version-specific references. There's an expectation that there's only one version of each resource instance in-scope for the publication process. Changing that will involve a significant refactoring of the publisher.
Jose Costa Teixeira (Mar 04 2020 at 19:11):
Yes, I understand that, and that was why I put identifier instead of reference:
I wanted to get around this issue by actually creating different resources with different identifiers and filenames, but where those names would have a syntax like resource#version.
The file system is ok with that name, but the IG publisher is not happy with having a resource with a '#'.
(which to me may mean it's a version but for the IG it should be a completely independent resource)
Grahame Grieve (Mar 04 2020 at 19:25):
....must be a valid identifier...
What's the exact error? I gt no matches for that text in the code
Jose Costa Teixeira (Mar 04 2020 at 19:35):
I just had a resource called dzop-team-002.v0 (filename and id) and it worked
Jose Costa Teixeira (Mar 04 2020 at 19:36):
now I renamed it to dzop-team-002#0 - changed the filename, the id, also changed the reference in the IG:
<resource> <reference> <reference value="CareTeam/dzop-team-0002#0"/> </reference>
Jose Costa Teixeira (Mar 04 2020 at 19:36):
now I get
Load Content (00:25.0276)
Publishing Content Failed: Bad Source Reference 'CareTeam/dzop-team-0002#0' - should have the format [Type]/[id] where id is a valid FHIR id type (00:27.0435)
(00:27.0446)
Use -? to get command line help (00:27.0449)
(00:27.0450)
Stack Dump (for debugging): (00:27.0469)
java.lang.Exception: Bad Source Reference 'CareTeam/dzop-team-0002#0' - should have the format [Type]/[id] where id is a valid FHIR id type
at org.hl7.fhir.igtools.publisher.SimpleFetcher.fetch(SimpleFetcher.java:181)
at org.hl7.fhir.igtools.publisher.Publisher.load(Publisher.java:2683)
at org.hl7.fhir.igtools.publisher.Publisher.createIg(Publisher.java:801)
at org.hl7.fhir.igtools.publisher.Publisher.execute(Publisher.java:666)
at org.hl7.fhir.igtools.publisher.Publisher.main(Publisher.java:6933)
Grahame Grieve (Mar 04 2020 at 19:56):
ok. that's the right message. it's from the publisher enforcing it's internal business rules
David Hay (Mar 04 2020 at 20:22):
'cause # is not valid in an id?
Grahame Grieve (Mar 04 2020 at 20:30):
right
Jose Costa Teixeira (Mar 10 2020 at 17:09):
just a thought: if # is not a valid id, can we use just that to signal the versions? in other words:
would it make sense to use it as a convention to name versions, and prepare the tooling to (gracefully) reject those ids, and eventually as tools catch up (if ever) then we keep the same convention?
Last updated: Apr 12 2022 at 19:14 UTC
