FHIR Chat · IG Release Publication procedure · IG creation

Stream: IG creation

Topic: IG Release Publication procedure


view this post on Zulip Jose Costa Teixeira (Mar 19 2021 at 21:47):

So, @ Bob Milius 's input was really helpful and unblocked me.
I can publish a first draft (to be corrected by anyone that knows better) of a simple procedure for gettting from A to B (similar to Bob's procedure but for an existing IG, and not only one freshly generated by sushi.

view this post on Zulip Jose Costa Teixeira (Mar 19 2021 at 21:49):

About Documentation: Confluence? A github repo with step by step progress?

view this post on Zulip Jose Costa Teixeira (Mar 19 2021 at 21:52):

I think that this deserves / needs a procedure. It's too sensitive to minor mistakes and some pre-processing would help.
My current understanding is the following (Taken from Grahame's idea):

  1. For every new IG, the user sets up the processing folder and dependencies
  2. When there is a new release, the user adds a token (a small json) which should the job the done
  3. When the user wants to roll back or remove a release, well, we'll have to see about that.

view this post on Zulip Jose Costa Teixeira (Mar 19 2021 at 21:53):

This sounds like a job for a GUI-enabled tool. A command line script should work, but would be ...meh...

view this post on Zulip Lloyd McKenzie (Mar 19 2021 at 22:40):

My leaning would be Confluence page

view this post on Zulip John Moehrke (Mar 22 2021 at 12:20):

yes, just document the steps on a confluence page.

view this post on Zulip Jose Costa Teixeira (Oct 01 2021 at 07:06):

Phase 2:
I want to start a process for semi-automating the release (the release is manually triggered and there's review needed, but the process steps for releasing can be automated).
I'm thinking of using github actions for that. Does anyone have ideas / requirements to share?

view this post on Zulip Jose Costa Teixeira (Oct 01 2021 at 07:11):

I'd start with:

  1. Release manager creates a RC branch
  2. Release manager Reviews IG in that branch, reviews QA, reviews a bunch of other stuff (manual work so we don't automate that)
  3. when done, the Release manager signs off by doing something special (not sure what, perhaps committing a release token) which triggers the process
  4. The publication process is executed. If all is fine (no errors), the github release is then created.

view this post on Zulip Eirik Myklebost (Oct 01 2021 at 08:23):

I've kind of done something similar. Some additional steps that might be applicable:

  1. Test the IG using example resources that should validate without errors, and invalid resources (negative-tests) that does not conform to the IG and is expected to trigger validation errors. I've created a lib called fhir-validator-junit-engine that might be of interest, the README contains example of how this can be integrated in a gh-action workflow.
  2. If you are planning to automate the publishing of FHIR Packages this can be done by including the package as an asset in the GitHub-release. The package-feed.xml can then be generated from the GitHub releases API using a template and hosted on e.g. gh-pages, here is an example of such a template using liquid, and here is an example of a gh-action.

view this post on Zulip Jose Costa Teixeira (Oct 01 2021 at 12:24):

Thanks @Eirik Myklebost for this input.

view this post on Zulip Jose Costa Teixeira (Oct 01 2021 at 12:25):

The validator - do you use it besides the FHIR validator that is run with the FHIR IG publisher?

view this post on Zulip Eirik Myklebost (Oct 01 2021 at 12:44):

Jose Costa Teixeira said:

The validator - do you use it besides the FHIR validator that is run with the FHIR IG publisher?

Yes. We use the junit-engine to get a TDD (test-driven-profiling if you'd like) feedback-loop while working locally on an IG. It is also used in pull-request checks for IGs to test validation of both valid and invalid resources, similar to how you would require unit-tests to pass before accepting a PR to a code project. These test resources are not necessarily included in the IG, especially when they are expected to be invalid according to the IG.

The IG Publisher is still doing validation, but that is part of the deployment step, after the tests have passed and the PR has been merged.

view this post on Zulip Jose Costa Teixeira (Oct 01 2021 at 12:56):

good. While I'm reading on this, let me ping @Jens Villadsen @João Almeida

view this post on Zulip Grahame Grieve (Oct 01 2021 at 19:37):

especially when they are expected to be invalid according to the IG

This has already come up. Perhaps the IG should explicitly have a way to include test cases that are in error?

view this post on Zulip Lloyd McKenzie (Oct 02 2021 at 01:32):

Maybe a flag that says "this is deliberately erroneous" that then allows the suppression of errors? - or better yet, spits out those errors in a fragment that can be rendered along with the example?

view this post on Zulip Jose Costa Teixeira (Oct 02 2021 at 08:30):

From the testing discussion - I was initially thinking of an IG parameter "skip_validation" with the urls of those resources that are to be accepted even if invalid.

view this post on Zulip Grahame Grieve (Oct 02 2021 at 10:47):

better to validate, and know that the errors are expected, and not errors in a different way

view this post on Zulip Jose Costa Teixeira (Oct 04 2021 at 23:44):

ok back to the procedure for publication, I think it could look likle this:
image.png

view this post on Zulip Jose Costa Teixeira (Oct 04 2021 at 23:49):

short:

  1. release manager (person) creates a github branch for a release candidate.
  2. after a first build, release manager does all the manual stuff (no automation on that for now)
  3. when all is ok, the release manager creates a release token which triggers the automatic publication for release (I have it scripted)
    If the script runs OK,

  4. an actual release is created (in github) with the package.tgz (and others?) as binaries

  5. optionally the release manager needs to create a PR for the updated ig-registry

view this post on Zulip Jose Costa Teixeira (Oct 04 2021 at 23:53):

The whole thing seems automatable (where it should) while ensuring that there is a manual trigger which is expected to be preceded by all the human review that is necessary

view this post on Zulip Eirik Myklebost (Oct 05 2021 at 11:49):

Do you have any thoughts\plans on automating the release-notes\changelog and versioning? E.g. google's release-please action.

view this post on Zulip Jose Costa Teixeira (Oct 05 2021 at 12:14):

I'll read up on this, thanks

view this post on Zulip Jose Costa Teixeira (Oct 05 2021 at 14:21):

For release notes, the only idea I have is that we could compile all the issues /changes that are closed in a release, and this would help create part of the release notes. Thus not automated , but assisted

view this post on Zulip Jose Costa Teixeira (Oct 05 2021 at 21:02):

(nevermind)

view this post on Zulip Jose Costa Teixeira (Oct 06 2021 at 04:49):

Is there a recommended / best branching strategy for releases?
a) when we need to make a release we make a Release Candidate branch. We clean up and release that one while the main branch is getting our future work.
b) While we are cleaning up a release we keep the main line clean, all the development goes into a development branch.

view this post on Zulip Jose Costa Teixeira (Oct 06 2021 at 04:50):

Any recommendations?

view this post on Zulip Jens Villadsen (Oct 10 2021 at 12:08):

There's a couple of flows that comes into mind: https://medium.com/@patrickporto/4-branching-workflows-for-git-30d0aaee7bf / https://www.atlassian.com/git/tutorials/comparing-workflows

view this post on Zulip Jens Villadsen (Oct 10 2021 at 12:09):

I know how I prefer my git branch strategy but that might not fit into other people's workflow

view this post on Zulip Jose Costa Teixeira (Oct 11 2021 at 22:12):

ok so my script will work independently of which branch. As long as there is an approval to release, it will do so.

view this post on Zulip Jose Costa Teixeira (Oct 11 2021 at 22:14):

@Grahame Grieve is it ok to make a change request to the Publisher to allow it to run without prompting for the y at the end?
(since I don't understand the -filter parameter, I don't dare make a PR)

view this post on Zulip Jose Costa Teixeira (Dec 22 2021 at 10:45):

One further step on this:
I took the assumption that the ImplementationGuide.url should be in the form
{canonical}/ImplementationGuide/{ig_id}

https://confluence.hl7.org/pages/viewpage.action?pageId=104580055#ProcessforPublishingaFHIRImplementationGuide(fornonHL7IGs)-2.SetuptheIGforpublication

view this post on Zulip Jose Costa Teixeira (Dec 22 2021 at 10:45):

TBH I only did that because that somehow worked.

view this post on Zulip Jose Costa Teixeira (Dec 22 2021 at 10:46):

Is this necessary? Is the ImplementationGuide's URL supposed to be in that form?

view this post on Zulip Jose Costa Teixeira (Dec 22 2021 at 10:54):

@Jens Villadsen this should matter to you

view this post on Zulip Jose Costa Teixeira (Dec 22 2021 at 10:54):

@Grahame Grieve I guess you're the one to help us here

view this post on Zulip Jose Costa Teixeira (Dec 22 2021 at 10:55):

@ Bob Milius just to say Hi on this :)

view this post on Zulip Grahame Grieve (Dec 22 2021 at 10:56):

yes that's how it should be

view this post on Zulip Jens Villadsen (Dec 22 2021 at 11:06):

Should that URL resolve?

view this post on Zulip Grahame Grieve (Dec 22 2021 at 11:09):

it generally should but it doesn't have to

view this post on Zulip Jens Villadsen (Dec 22 2021 at 12:00):

And what should it resolve to? The overall page of the ig / history page / something else?

view this post on Zulip Jens Villadsen (Dec 22 2021 at 12:02):

Is there a good example where it resolves?

view this post on Zulip Grahame Grieve (Dec 22 2021 at 12:02):

apparently not

view this post on Zulip Grahame Grieve (Dec 22 2021 at 12:03):

I

view this post on Zulip Grahame Grieve (Dec 22 2021 at 12:03):

I'll have to investigate that one

view this post on Zulip Jens Villadsen (Dec 22 2021 at 12:06):

@Jose Costa Teixeira could you perhaps also look into why the release script puts importance into the name of the folder where it is executed from?

view this post on Zulip Jens Villadsen (Dec 22 2021 at 12:08):

@Grahame Grieve if noone uses the feature of the IG's URL they way is it currently constructed, why then have it at all?

view this post on Zulip Jens Villadsen (Dec 22 2021 at 12:10):

@Jose Costa Teixeira / @Grahame Grieve this is current issue (that hits me and probably others) that is probably triggered by implied naming convention:

Update /Users/jvi/hl7.fhir.dk.core/www for http://hl7.dk/fhir/core/ImplementationGuide/hl7.fhir.dk.core
Error in /Users/jvi/hl7.fhir.dk.core/www/package-list.json:path to build.fhir.org should not use https://
  /Users/jvi/hl7.fhir.dk.core/www/1.1.0: This page is part of the HL7 FHIR Implementation Guide: DK Core (v1.1.0: Release) based on <a href="http://hl7.org/fhir/R4">FHIR R4</a>. This is the current published version in its permanent home (it will always be available at this URL).  For a full list of available versions, see the <a href="http://hl7.dk/fhir/core/ImplementationGuide/hl7.fhir.dk.core/history.html">Directory of published versions <img src="external.png" style="text-align: baseline"></a>
  .. 282 files checked, 282 updated
  .. 101 clones checked, 202 updated
Problem #15 with /Users/jvi/hl7.fhir.dk.core/www/1.1.0/package.tgz: expected canonical http://hl7.dk/fhir/core/ImplementationGuide/hl7.fhir.dk.core but found http://hl7.dk/fhir/core
  .. 0 redirections (0 created/updated)
  /Users/jvi/hl7.fhir.dk.core/www: This page is part of the HL7 FHIR Implementation Guide: DK Core (v1.1.0: Release) based on <a href="http://hl7.org/fhir/R4">FHIR R4</a>. This is the current published version.  For a full list of available versions, see the <a href="http://hl7.dk/fhir/core/ImplementationGuide/hl7.fhir.dk.core/history.html">Directory of published versions <img src="external.png" style="text-align: baseline"></a>
  .. 282 files checked, 282 updated
  .. 204 clones checked, 0 updated
Problem #15 with /Users/jvi/hl7.fhir.dk.core/www/package.tgz: expected canonical http://hl7.dk/fhir/core/ImplementationGuide/hl7.fhir.dk.core but found http://hl7.dk/fhir/core
  .. 0 redirections (0 created/updated)
java.io.IOException: Source '/Users/jvi/hl7.fhir.dk.core/www/dist-hist' and destination '/Users/jvi/hl7.fhir.dk.core/www/dist-hist' are the same
    at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:1351)
    at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:1246)
    at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:1215)
    at org.hl7.fhir.igtools.publisher.utils.IGReleaseUpdater.checkCopyFolderFromRoot(IGReleaseUpdater.java:261)
    at org.hl7.fhir.igtools.publisher.utils.IGReleaseUpdater.check(IGReleaseUpdater.java:216)
    at org.hl7.fhir.igtools.publisher.utils.IGWebSiteMaintainer.execute(IGWebSiteMaintainer.java:106)
    at org.hl7.fhir.igtools.publisher.Publisher.main(Publisher.java:8743)

    Source '/Users/jvi/hl7.fhir.dk.core/www/dist-hist' and destination '/Users/jvi/hl7.fhir.dk.core/www/dist-hist' are the same

view this post on Zulip Jose Costa Teixeira (Dec 22 2021 at 12:28):

right. This dist-hist thing happens every time I run the procedure, but I don't know where the folders should be to avoid it

view this post on Zulip Jose Costa Teixeira (Dec 22 2021 at 12:31):

@Jens Villadsen the release script i made no longer enforces the folder name to be the same as ig id (i forgot why it was there, so we may have some surprises ahead)

view this post on Zulip Jens Villadsen (Dec 22 2021 at 17:42):

@Jose Costa Teixeira where's the 'normative' version of the script?

view this post on Zulip Jose Costa Teixeira (Dec 22 2021 at 18:06):

I haven't released it formally yet..

view this post on Zulip Jose Costa Teixeira (Dec 22 2021 at 18:07):

but you can see it here
https://github.com/hl7-be/IG-release-pub/blob/master/release.py

view this post on Zulip Grahame Grieve (Jan 06 2022 at 05:05):

umm I don't understand this. dist-hist should be at the root, and will just be copied as necessary

view this post on Zulip Jose Costa Teixeira (Jan 06 2022 at 11:44):

dist-hist is not in <root>, but in the folder <root>/fhir-ig-history-templatewhich is where the history template is cloned to

view this post on Zulip Jose Costa Teixeira (Jan 06 2022 at 11:45):

Is this the right place or do we have to copy the folder somewhere?

view this post on Zulip Grahame Grieve (Jan 06 2022 at 18:01):

cloned to?

view this post on Zulip Jose Costa Teixeira (Jan 06 2022 at 20:41):

procedure is:
...

  1. Change the working folder to the IG root folder
    cd my-ig

  2. Create a folder that will be your web root (typically the folder that you will publish to the web)
    md www

  3. Copy content of history template to web root folder
    xcopy /s /y ../fhir-ig-history-template/* www

view this post on Zulip Grahame Grieve (Jan 06 2022 at 20:45):

I don't think that 8 is correct.

view this post on Zulip Grahame Grieve (Jan 06 2022 at 20:45):

you need the history template, but you just pass that as a reference to the publish process - it should be somewhere else

view this post on Zulip Jose Costa Teixeira (Jan 06 2022 at 21:07):

this ?
java -jar ../publisher.jar -publish-update -folder www -registry ../ig-registry/fhir-ig-list.json -history ../fhir-ig-history-template

view this post on Zulip Grahame Grieve (Jan 06 2022 at 21:08):

yes

view this post on Zulip Jose Costa Teixeira (Jan 06 2022 at 21:08):

(which actually points to ../fhir-ig-history-template), so point 8 seems useless)

view this post on Zulip Jose Costa Teixeira (Jan 06 2022 at 21:09):

but this is the procedure that causes the issue

view this post on Zulip Jose Costa Teixeira (Jan 06 2022 at 21:10):

in other words, if I run that line which points to ../fhir-ig-history-template, i get the error "source dist-hist and target are the same.."

view this post on Zulip Grahame Grieve (Jan 06 2022 at 21:22):

because it has to be somewhere else

view this post on Zulip Jose Costa Teixeira (Jan 06 2022 at 21:23):

any idea where i should put it?

view this post on Zulip Grahame Grieve (Jan 06 2022 at 21:23):

anywhere else

view this post on Zulip Jose Costa Teixeira (Jan 06 2022 at 21:39):

I get this
History Error: C:\work\ImplementationGuides\dk-core\www\dist-hist doe not exist
History Error: C:\work\ImplementationGuides\dk-core\www\assets-hist doe not exist
when running
java -jar ../publisher.jar -publish-update -folder www -registry ../ig-registry/fhir-ig-list.json -history ./anywhereelse

view this post on Zulip Jose Costa Teixeira (Jan 06 2022 at 21:45):

(after copying the content of the history template to ./anywhereelse):

view this post on Zulip Jose Costa Teixeira (Jan 06 2022 at 21:45):

image.png

view this post on Zulip Grahame Grieve (Jan 07 2022 at 04:46):

@Manisha Mantri this is the thread you want to know

view this post on Zulip Grahame Grieve (Jan 07 2022 at 04:46):

@Sayali Pophalkar

view this post on Zulip Manisha Mantri (Jan 07 2022 at 06:22):

Thank you

view this post on Zulip Jose Costa Teixeira (Jan 07 2022 at 11:04):

@Grahame Grieve any advice on how to fix the error I get?

view this post on Zulip Jens Villadsen (Jan 11 2022 at 12:45):

@Jose Costa Teixeira - there is still something funky about the links generated when publishing. For instance when looking at https://docs.ehealth.sundhed.dk/latest/ig/StructureDefinition-ehealth-patient.html#profile ( and IG that depens on the DK core)-
image.png

view this post on Zulip Lloyd McKenzie (Jan 11 2022 at 14:59):

I suspect someone put the full URL for the IG somewhere they should have put the canonical URL for the IG root.

view this post on Zulip Jens Villadsen (Jan 11 2022 at 17:13):

That must be the publish script I assume @Jose Costa Teixeira ?

view this post on Zulip Grahame Grieve (Jan 11 2022 at 17:15):

don't see how it could be. This comes from the IG itself

view this post on Zulip Jens Villadsen (Jan 11 2022 at 17:17):

https://github.com/hl7dk/dk-core/tree/1.1.0 is the state of the IG when I ran the script

view this post on Zulip Jens Villadsen (Jan 11 2022 at 17:18):

I also see that the web base on the qa page has the same wrong address

view this post on Zulip Jens Villadsen (Jan 11 2022 at 17:20):

The script being https://github.com/hl7dk/dk-core/blob/1.1.0/release.py

view this post on Zulip Jens Villadsen (Jan 11 2022 at 18:05):

@Grahame Grieve are you saying you dont think it is a bug?

view this post on Zulip Grahame Grieve (Jan 11 2022 at 18:05):

it's a bug somewhere. But where?

view this post on Zulip Jens Villadsen (Jan 11 2022 at 18:09):

I hope to catch @Jose Costa Teixeira for a moment and see if he has a clue

view this post on Zulip Jose Costa Teixeira (Jan 11 2022 at 18:35):

Sorry for the delay. From what I can see, the issue is visible not in the dk-core, but on an ig that depends on dk-core, right?

view this post on Zulip Jose Costa Teixeira (Jan 11 2022 at 18:35):

@Jens Villadsen

view this post on Zulip Jose Costa Teixeira (Jan 11 2022 at 18:37):

the dk-core ig seems ok (unless I'm missing sometyhing)

view this post on Zulip Jens Villadsen (Jan 11 2022 at 18:38):

Yes and no

view this post on Zulip Jens Villadsen (Jan 11 2022 at 18:38):

It looks correct from the inside

view this post on Zulip Jens Villadsen (Jan 11 2022 at 18:39):

But when you depend it there is yet another warning

view this post on Zulip Jens Villadsen (Jan 11 2022 at 18:51):

What ... ?
Publishing Content Failed: Cannot invoke "com.google.gson.JsonElement.getAsString()" because the return value of "com.google.gson.JsonObject.get(String)" is null (00:46.0809) - that one is for you @Grahame Grieve

view this post on Zulip Jens Villadsen (Jan 11 2022 at 18:52):

happens when running genonceon the DK Core IG

view this post on Zulip Jens Villadsen (Jan 11 2022 at 18:52):

using IG Pub 1.1.94 (latest stable)

view this post on Zulip Jens Villadsen (Jan 11 2022 at 19:02):

wait ... my bad ... wrong directory

view this post on Zulip Jens Villadsen (Jan 11 2022 at 19:06):

nope ... it was it fact the right directory

view this post on Zulip Jens Villadsen (Jan 11 2022 at 19:06):

The bug is real - and the following error is also present Error setting locale for jurisdiction: Cannot invoke "String.toUpperCase()" because "c" is null (00:15.0436)

view this post on Zulip Jose Costa Teixeira (Jan 11 2022 at 19:10):

i'm lost

view this post on Zulip Jose Costa Teixeira (Jan 11 2022 at 19:10):

how do i replicate this on my machine?

view this post on Zulip Jose Costa Teixeira (Jan 11 2022 at 19:11):

(the implementationguide/hl7.fhir.dk.core thing)

view this post on Zulip Jens Villadsen (Jan 11 2022 at 19:12):

2 sec

view this post on Zulip Jens Villadsen (Jan 11 2022 at 19:15):

ok ... if you run python3 release.py release.jsonfrom the HEAD of the master branch

view this post on Zulip Jens Villadsen (Jan 11 2022 at 19:16):

The package-feed.xml will look like the following:

<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:fhir="http://hl7.org/fhir/feed" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0">
  <channel>
    <title>HL7DK FHIR Packages</title>
    <description>New Packages published by HL7DK></description>
    <link>http://hl7.dk/fhir/corehl7.fhir.dk.core/package-feed.xml</link>
    <generator>HL7, Inc FHIR Publication tooling</generator>
    <lastBuildDate>Sun, 19 Dec 2021 09:10:14 GMT</lastBuildDate>
    <atom:link href="http://hl7.dk/fhir/corehl7.fhir.dk.core/package-feed.xml" rel="self" type="application/rss+xml"/>
    <pubDate>Sun, 19 Dec 2021 09:10:14</pubDate>
    <language>en></language>
    <ttl>600</ttl>
    <item>
      <title>hl7.fhir.dk.core#1.1.0</title>
      <description>A FHIR Implementation Guide for the Danish common needs across healthcare sectors (built Sun, Dec 19, 2021 21:09+0100+01:00)</description>
      <link>http://hl7.dk/fhir/corehl7.fhir.dk.core/1.1.0/package.tgz</link>
      <guid isPermaLink="true">http://hl7.dk/fhir/corehl7.fhir.dk.core/1.1.0/package.tgz</guid>
      <dc:creator>HL7DK</dc:creator>
      <fhir:version>4.0.1</fhir:version>
      <fhir:kind>fhir.ig</fhir:kind>
      <pubDate>Sat, 18 Dec 2021 12:00:00 GMT</pubDate>
    </item>
  </channel>
</rss>

view this post on Zulip Jens Villadsen (Jan 11 2022 at 19:16):

The http://hl7.dk/fhir/corehl7.fhir.dk.core/package-feed.xml is missing a /

view this post on Zulip Jens Villadsen (Jan 11 2022 at 19:17):

now I can't tell if any fixes in the IGPublisher since the publication has fixed the /ImplementationGuide/<id>

view this post on Zulip Jens Villadsen (Jan 11 2022 at 19:19):

and http://hl7.dk/fhir/corehl7.fhir.dk.core/package-feed.xml is not correct either ... The link should simply be http://hl7.dk/fhir/coreas that is the base URL

view this post on Zulip Jens Villadsen (Jan 11 2022 at 19:20):

so it looks like the ìd`is appended when it should not be appended at all

view this post on Zulip Jens Villadsen (Jan 11 2022 at 19:21):

You follow @Jose Costa Teixeira ?

view this post on Zulip Jens Villadsen (Jan 11 2022 at 19:44):

and @Jose Costa Teixeira another thing - the publish script still insist on that the folder it runs in should have the name of the Id of the IG ... :thinking:

view this post on Zulip Elliot Silver (Jan 11 2022 at 19:56):

Jens Villadsen said:

The bug is real - and the following error is also present Error setting locale for jurisdiction: Cannot invoke "String.toUpperCase()" because "c" is null (00:15.0436)

I just noticed this on my build too. From the debugging I could to, it doesn't like my jurisdiction:

  <jurisdiction>
    <coding>
      <system value="urn:iso:std:iso:3166"/>
      <code value="CA"/>
    </coding>
  </jurisdiction>

It's trying to find CA in iso:3166:-2 (3166:-2 is subnational jurisdictions, not 2-letter jurisdictions.)

I don't think that's your larger problem though.

view this post on Zulip Jose Costa Teixeira (Jan 11 2022 at 19:56):

Jens Villadsen said:

and Jose Costa Teixeira another thing - the publish script still insist on that the folder it runs in should have the name of the Id of the IG ... :thinking:

you didn't have the fix in your branch. I made a PR now

view this post on Zulip Jose Costa Teixeira (Jan 11 2022 at 19:59):

(made the PR in the 1.1.0 branch)

view this post on Zulip Jose Costa Teixeira (Jan 11 2022 at 20:02):

for the rest, the trailing / etc - can i look into this tomorrow? I am struggling to finish some slides for a session I'm hosting tomorrow

view this post on Zulip Jens Villadsen (Jan 11 2022 at 20:09):

Sure - just happy to receive help

view this post on Zulip Jens Villadsen (Jan 11 2022 at 20:10):

Arh ... The PR is targeted 1.1.0 and not master but I can fix it locally

view this post on Zulip Grahame Grieve (Jan 12 2022 at 00:24):

ok. so let's have some documentation: https://confluence.hl7.org/display/FHIR/Maintaining+a+FHIR+IG+Publication

view this post on Zulip Grahame Grieve (Jan 12 2022 at 00:24):

comments welcome

view this post on Zulip Elliot Silver (Jan 12 2022 at 01:28):

Thank you!

view this post on Zulip Jens Villadsen (Jan 12 2022 at 08:21):

@Jose Costa Teixeira let me know if I can assist you any time today debugging

view this post on Zulip Jose Costa Teixeira (Jan 12 2022 at 09:18):

Thanks @Grahame Grieve for this.

view this post on Zulip Jose Costa Teixeira (Jan 12 2022 at 09:19):

@Jens Villadsen After my webinar today i'll use Grahame's notes and will step through the script, updating where needed. Want to do it together, or you want me to suffer the first impact and then you jump in?

view this post on Zulip Jens Villadsen (Jan 12 2022 at 10:15):

That depends on when your webinar ends

view this post on Zulip Jose Costa Teixeira (Jan 12 2022 at 10:37):

I meant _as of_ , not immediately after

view this post on Zulip Jens Villadsen (Jan 12 2022 at 13:42):

You got a rough point in time?

view this post on Zulip Mark Scrimshire (Jan 12 2022 at 15:26):

I'm getting the same error on PDex
Error setting locale for jurisdiction: Cannot invoke "String.toUpperCase()" because "c" is null (00:46.0593)
Processing Conformance Resources (00:46.0594)
Generating Snapshots (00:46.0854)
Generating Narratives (00:47.0352)
Validating Conformance Resources (01:25.0239)
Publishing Content Failed: Error parsing JSON source: Unexpected char '<' in json stream at Line 1 (path=[null]) (01:32.0469)

view this post on Zulip Jose Costa Teixeira (Jan 12 2022 at 16:11):

@Jens Villadsen since I missed the vilcan track to show my resources and the next session is only in 1 hour, I'm getting into this now

view this post on Zulip Linda (Jan 12 2022 at 16:27):

seeing same in RA

view this post on Zulip Jose Costa Teixeira (Jan 12 2022 at 16:43):

@Grahame Grieve using the go-publish approach brings back the issues I had with the canonical urls - it says canonical URL of https://costateixeira.github.io/hl7.fhir.be.core does not match the required canonical of http://hl7.org/fhir/be/core

view this post on Zulip Grahame Grieve (Jan 12 2022 at 17:21):

right. that's an actual issue.

view this post on Zulip Grahame Grieve (Jan 12 2022 at 17:21):

why aren't you publishing at the canonical URL?

view this post on Zulip Grahame Grieve (Jan 12 2022 at 17:30):

Error setting locale for jurisdiction: Cannot invoke "String.toUpperCase()" because "c" is null (00:46.0593)

A bunch of people have reported this, but I just can't reproduce this ?

view this post on Zulip Jose Costa Teixeira (Jan 12 2022 at 17:31):

Grahame Grieve said:

why aren't you publishing at the canonical URL?

Where do we set the required canonical URL? I'm looking at non-HL7 IGs

view this post on Zulip Grahame Grieve (Jan 12 2022 at 17:33):

you define it

view this post on Zulip Elliot Silver (Jan 12 2022 at 17:36):

Grahame Grieve said:

Error setting locale for jurisdiction: Cannot invoke "String.toUpperCase()" because "c" is null (00:46.0593)

A bunch of people have reported this, but I just can't reproduce this ?

I investigated a bit, and you're validating against iso 3166:-2, not 3166.

I see it in the test repo I sent you yesterday.

view this post on Zulip Grahame Grieve (Jan 12 2022 at 17:37):

like I said, I can't reproduce it

view this post on Zulip Jose Costa Teixeira (Jan 12 2022 at 17:38):

Grahame Grieve said:

you define it

Where?

view this post on Zulip Jose Costa Teixeira (Jan 12 2022 at 17:39):

(maybe I should just acknowledge the thread has been hijacked and start a new one..?)

view this post on Zulip Grahame Grieve (Jan 12 2022 at 17:41):

the canonical is in the npm package, based on what the IG defines

view this post on Zulip Jose Costa Teixeira (Jan 12 2022 at 18:08):

the URLs (ig.,url, ig.canonical and package-list) are all the same

view this post on Zulip Grahame Grieve (Jan 12 2022 at 18:08):

what are they?

view this post on Zulip Jose Costa Teixeira (Jan 12 2022 at 18:08):

maybe I need to build the ig first to set the npm package right

view this post on Zulip Jose Costa Teixeira (Jan 12 2022 at 18:09):

the url is
https://costateixeira.github.io/hl7.fhir.be.core

view this post on Zulip Grahame Grieve (Jan 12 2022 at 18:18):

yes that's true, you do need to do a local build with the URLs correct.

view this post on Zulip Grahame Grieve (Jan 12 2022 at 18:19):

because the QA checks that run first assume that these things are all correct and you've run a local build to verify that everything is building correctly (per documented process)

view this post on Zulip Jose Costa Teixeira (Jan 12 2022 at 18:59):

still same issue.

view this post on Zulip Jose Costa Teixeira (Jan 12 2022 at 19:00):

my package.json inside package.tgz says
"canonical": "https://costateixeira.github.io/hl7.fhir.be.core",

view this post on Zulip Jose Costa Teixeira (Jan 12 2022 at 19:05):

I think it may be the java code that's doing this. I see no reference to hl7.org/fhir/be/ in the artifacts

view this post on Zulip Jens Villadsen (Jan 12 2022 at 21:05):

@Jose Costa Teixeira can I be of any help?

view this post on Zulip Jose Costa Teixeira (Jan 12 2022 at 21:07):

If you want to go through the process that Grahame described... You also have an affiliate IG so it could be useful to see if you end up with similar result

view this post on Zulip Jose Costa Teixeira (Jan 12 2022 at 21:10):

)Perhaps the problem is in my brain. But I don't expect my brain to have issues, it's almost brand new and rarely used.)

view this post on Zulip Jens Villadsen (Jan 12 2022 at 21:54):

Well this is still broken: https://chat.fhir.org/#narrow/stream/179252-IG-creation/topic/IG.20Release.20Publication.20procedure/near/267631188

view this post on Zulip Jens Villadsen (Jan 12 2022 at 23:00):

@Jose Costa Teixeira have you debugged this part? https://github.com/hl7dk/dk-core/blob/4216254684cec264a1d61a8189244017f67dff08/release.py#L156

view this post on Zulip Jens Villadsen (Jan 12 2022 at 23:01):

It doesnt look correct to me

view this post on Zulip Jose Costa Teixeira (Jan 12 2022 at 23:36):

That code was a way to get the canonical URL from the IG and also check that the URL contains the ig_id/ImplementationGuide/ig_id pattern. It does that job but is not clear to understand or change. I'll improve that

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 00:43):

Jose Costa Teixeira said:

my package.json inside package.tgz says
"canonical": "https://costateixeira.github.io/hl7.fhir.be.core",

@Grahame Grieve Still same issue - something seems to be forcing for a hl7.org url

view this post on Zulip Jens Villadsen (Jan 13 2022 at 10:04):

Well ... there are some assumptions in the computations that do not hold

view this post on Zulip Jens Villadsen (Jan 13 2022 at 10:05):

The IG url that is fetched on https://github.com/hl7dk/dk-core/blob/4216254684cec264a1d61a8189244017f67dff08/release.py#L157 from the IG resource in the DK core ends up being http://hl7.dk/fhir/core/ImplementationGuide/hl7.fhir.dk.core

view this post on Zulip Jens Villadsen (Jan 13 2022 at 10:06):

On line 163 that ends up assigning the base to http://hl7.dk/fhir/core/ImplementationGuide/which should have been http://hl7.dk/fhir/core

view this post on Zulip Jens Villadsen (Jan 13 2022 at 10:06):

I would think (depending on the meaning of base)

view this post on Zulip Jens Villadsen (Jan 13 2022 at 10:09):

As shown in the QA page for eg. https://build.fhir.org/ig/hl7dk/dk-core/qa.html - The canonical should be http://hl7.dk/fhir/coreand the package-id should be `hl7.fhir.dk.core. Package-id seems to be correct for the time being

view this post on Zulip Jens Villadsen (Jan 13 2022 at 10:31):

So ... going back to the drawing board ...

view this post on Zulip Jens Villadsen (Jan 13 2022 at 10:32):

The DK QA page says (as I expected, correctly) that the Canonical is http://hl7.dk/fhir/core

view this post on Zulip Jens Villadsen (Jan 13 2022 at 10:32):

The documentation states:
image.png

view this post on Zulip Jens Villadsen (Jan 13 2022 at 10:33):

So ... when the FSH calculated ImplementationGuide.url ends up being (which all the calculations afterwards depends on):
image.png

view this post on Zulip Jens Villadsen (Jan 13 2022 at 10:33):

Is that then incorrect?

view this post on Zulip Jens Villadsen (Jan 13 2022 at 10:34):

Shoulnd't that url just have been http://hl7.dk/fhir/core ?

view this post on Zulip Jens Villadsen (Jan 13 2022 at 10:48):

So @Jose Costa Teixeira if I'm correct - the ig_canonical should be calculated as this: ig_canonical = url_data[0].split('/ImplementationGuide/')[0]

view this post on Zulip Jens Villadsen (Jan 13 2022 at 10:49):

If I add the following logging around line 240-ish

        print ("URL in ig.ini does not match that of the IG")
        print ("URL in ig.ini:" + publishini['website']['url'])
        print ("ig_canonical: " + ig_canonical )

view this post on Zulip Jens Villadsen (Jan 13 2022 at 10:50):

I get this:

### Reading and checking publish.ini
URL in ig.ini does not match that of the IG
URL in ig.ini:http://hl7.dk/fhir/core/ImplementationGuide/hl7.fhir.dk.core
ig_canonical: http://hl7.dk/fhir/core

view this post on Zulip Jens Villadsen (Jan 13 2022 at 10:52):

now, the publish.ini is autogenerated and reflects the URL in the ImplementationGuide.url resource - which I'm more and more convinced is wrong

view this post on Zulip Jens Villadsen (Jan 13 2022 at 10:53):

Now - someone tell me I'm wrong.

view this post on Zulip Chris Moesel (Jan 13 2022 at 13:55):

Jens Villadsen said:

So ... when the FSH calculated ImplementationGuide.url ends up being (which all the calculations afterwards depends on):
image.png

I haven't followed this whole conversation closely enough to know what's really going on (sorry)! But... I think SUSHI does the right thing here. There is a difference between the canonical base for an IG and the canonical URL for the IG resource. And I think it is correct for the canonical URL of the IG resource to end in /ImplementationGuide/${packageId}. I think you'll find this pattern in nearly all (if not ALL) HL7 IGs. For example, US Core: http://www.hl7.org/fhir/us/core/ImplementationGuide-hl7.fhir.us.core.html.

view this post on Zulip Jens Villadsen (Jan 13 2022 at 14:11):

@Chris Moesel Yeah - I've seen the same pattern for the other Ig's as well so I agree with you

view this post on Zulip Jens Villadsen (Jan 13 2022 at 14:12):

That then brings we to the question: Is it the case that the canonical base is only indirectly included in the ImplementationGuide resource?

view this post on Zulip Jens Villadsen (Jan 13 2022 at 14:14):

And if you are correct - (which I believe you are) - I think the documentation on the ImplementationGuide.url should be a bit more precise in its wording or at least state an example:

An absolute URI that is used to identify this implementation guide when it is referenced in a specification, model, design or an instance; also called its canonical identifier. This SHOULD be globally unique and SHOULD be a literal address at which at which an authoritative instance of this implementation guide is (or will be) published. This URL can be the target of a canonical reference. It SHALL remain the same when the implementation guide is stored on different servers.

view this post on Zulip Vassil Peytchev (Jan 13 2022 at 14:16):

AFAIK, there is no requirement that the canonical ends with ${packageId}. It just ends with the ImplementationGuide.id from the ImplementationGuide resource.

See https://build.fhir.org/ig/HL7/fhir-ipa/branches/master/qa.html

view this post on Zulip Chris Moesel (Jan 13 2022 at 14:38):

@Vassil Peytchev - I may have been overly prescriptive. You're right that the id in the URI needs to match the id on the ImplementationGuide resource. That is what's most important -- so thanks for the correction. Perhaps having the IG resource id match the package id is just a best practice? I know I've definitely seen it a lot.

view this post on Zulip Vassil Peytchev (Jan 13 2022 at 15:07):

I don't think it is a best practice for all cases. From what I understood, for HL7 published IGs, the ImplementationGuide.id must match the code in the approval document, and that is usually not the packageId.

view this post on Zulip Jens Villadsen (Jan 13 2022 at 17:36):

How is the URL that goes into publish.ini supposed to look like?

view this post on Zulip Jens Villadsen (Jan 13 2022 at 17:42):

Should that be the canonical or the value of ImplementationGuide.url

view this post on Zulip Vassil Peytchev (Jan 13 2022 at 17:55):

The value of ImplementationGuide.url is the canonical of the IG.

view this post on Zulip Jens Villadsen (Jan 13 2022 at 18:18):

Okay ...

view this post on Zulip Jens Villadsen (Jan 13 2022 at 18:28):

Well ... https://fshschool.org/docs/sushi/configuration/ says that the canonical here is the canonical URL - not the base

view this post on Zulip Jens Villadsen (Jan 13 2022 at 18:28):

Right?

view this post on Zulip Jens Villadsen (Jan 13 2022 at 19:01):

@Chris Moesel

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 19:10):

afaik sushi does assume that the IG's id is the same as the package id but we can override that by specifying both the id and the packageid

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 19:11):

the canonical I've seen documented and has worked (kind of) so far is the one with the full canonical path of the implementationguide resource, not the base url for the canonicals in the IG's resources

view this post on Zulip Jens Villadsen (Jan 13 2022 at 20:12):

Regardless - @Jose Costa Teixeira the publish script currently stops on https://github.com/hl7dk/dk-core/blob/master/release.py#L239

view this post on Zulip Jens Villadsen (Jan 13 2022 at 20:13):

Due to the fact that the content of the generated publish.ini contains:

[website]
style = fhir.layout
server = apache
url = http://hl7.dk/fhir/corehl7.fhir.dk.core
org = HL7DK
no-registry = 1

[feeds]
package = package-feed.xml
publication = publication-feed.xml

view this post on Zulip Grahame Grieve (Jan 13 2022 at 20:13):

finally getting to look at this...

view this post on Zulip Grahame Grieve (Jan 13 2022 at 20:14):

the publication process gets the canonical from the npm package

view this post on Zulip Grahame Grieve (Jan 13 2022 at 20:15):

then it checks the canonical against the package id using some business rules that I think apply for HL7 publications. If they are wrong for some affiliates, we can discuss that.

view this post on Zulip Grahame Grieve (Jan 13 2022 at 20:16):

then it loads the package-list from the destination and cross-checks all the canonicals and package ids

view this post on Zulip Grahame Grieve (Jan 13 2022 at 20:16):

and it also checks that the websites publish.ini matches the canonical URL

view this post on Zulip Grahame Grieve (Jan 13 2022 at 20:16):

@Jens Villadsen that publish.ini is not correct

view this post on Zulip Jens Villadsen (Jan 13 2022 at 20:17):

its generated ...

view this post on Zulip Jens Villadsen (Jan 13 2022 at 20:17):

by this: https://github.com/hl7dk/dk-core/blob/4216254684cec264a1d61a8189244017f67dff08/release.py#L217

view this post on Zulip Grahame Grieve (Jan 13 2022 at 20:17):

the publish.ini url is where the root of the folder is published - I would expect it to be http://hl7.dk/fhir

view this post on Zulip Jens Villadsen (Jan 13 2022 at 20:18):

that sounds about right - so what did I specify incorrectly?

view this post on Zulip Chris Moesel (Jan 13 2022 at 20:21):

Jens Villadsen said:

Well ... https://fshschool.org/docs/sushi/configuration/ says that the canonical here is the canonical URL - not the base.
Right?

That's probably just imprecise language on our part. The intention is that the canonical field in the config (or in sushi-config.yaml) is really the base canonical for the IG (as opposed to the canonical of the IG resource). Sorry for any confusion there.

view this post on Zulip Grahame Grieve (Jan 13 2022 at 20:23):

there may be some confusion in the spec here - I originally just used the base canonical, but @Lloyd McKenzie insisted (rightly :sad: ) that the canonical URL for the IG resource is not the same as the canonical base for the IG as a whole. So as long as I have any say - which is in all the tools - there'll be an algorithmic relationship between the two, since it's the base one that I care about

view this post on Zulip Jens Villadsen (Jan 13 2022 at 20:28):

May I suggest that you add examples in the doc where possible since the language is not clear

view this post on Zulip Grahame Grieve (Jan 13 2022 at 20:30):

which doc?

view this post on Zulip Jens Villadsen (Jan 13 2022 at 20:30):

for starters - the fsh configuration page

view this post on Zulip Jens Villadsen (Jan 13 2022 at 20:32):

Grahame Grieve said:

the publish.ini url is where the root of the folder is published - I would expect it to be http://hl7.dk/fhir

where is it possible to specify the root folder?

view this post on Zulip Grahame Grieve (Jan 13 2022 at 20:33):

in the publish.ini

view this post on Zulip Jens Villadsen (Jan 13 2022 at 20:34):

aint that a product of running https://github.com/hl7dk/dk-core/blob/4216254684cec264a1d61a8189244017f67dff08/release.py#L217 ?

view this post on Zulip Grahame Grieve (Jan 13 2022 at 20:35):

I don't know what that py file is, but for me, publish.ini is an independent input - why generate it?

view this post on Zulip Grahame Grieve (Jan 13 2022 at 20:36):

nothing in there can be inferred from anywhere else

view this post on Zulip Jens Villadsen (Jan 13 2022 at 20:36):

that explains some parts ... @Jose Costa Teixeira - looks like we still got work to do

view this post on Zulip Jens Villadsen (Jan 13 2022 at 20:37):

So this web IG root should probably go into the release.json

view this post on Zulip Grahame Grieve (Jan 13 2022 at 20:38):

why?

view this post on Zulip Grahame Grieve (Jan 13 2022 at 20:39):

it's a property of the website, nothing to do with any release

view this post on Zulip Jens Villadsen (Jan 13 2022 at 20:39):

the release.json is the parameter file that is used to control the python script

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 20:42):

I'm looking at this now

view this post on Zulip Grahame Grieve (Jan 13 2022 at 20:44):

I don't know what the python script is doing, but publish.ini is input into any automated process, not output

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 20:44):

Grahame Grieve said:

I don't know what the python script is doing, but publish.ini is input into any automated process, not output

indeed

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 20:44):

perhaps useful to separate a few things

view this post on Zulip Jens Villadsen (Jan 13 2022 at 20:44):

@Jose Costa Teixeira where in the script is it publish.ini generated?

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 20:45):

the python script is not an official thing (yet? - but there would be several discussions for that). It's a way to automate the procedure described by Grahame. Automation not to save effort, but to avoid variance

view this post on Zulip Jens Villadsen (Jan 13 2022 at 20:46):

if it can be used by the belgium affiliate and used by the danish affiliate ...

view this post on Zulip Jens Villadsen (Jan 13 2022 at 20:46):

i would say it can be used by any affiliate

view this post on Zulip Grahame Grieve (Jan 13 2022 at 20:46):

what does it automate?

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 20:46):

(right, we'll get there. let me type a few more things)

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 20:47):

the python script takes a "release token" that contains all the information that is needed for the procedure - namely the data that would be needed to say "hey, there's an IG, take this data and go make a release".

view this post on Zulip Elliot Silver (Jan 13 2022 at 20:47):

It would be helpful to document the applicable canonical better for package-list.json and package.json.

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 20:48):

this is the release.json. release.json is a proposal. I think it makes sense at least in this phase but hopefully in the long run, if we agree that we could automate the release build (only the build part)

view this post on Zulip Grahame Grieve (Jan 13 2022 at 20:49):

well, I do need to automate this for HL7. But what I need to automate is:

view this post on Zulip Grahame Grieve (Jan 13 2022 at 20:49):

  • update the local copy of the website
  • execute the -go-publish process
  • update the changes to the website

view this post on Zulip Jens Villadsen (Jan 13 2022 at 20:50):

Before you type anything more ... a wild guess would be that the requirements by HL7 org is beyond affiliate needs

view this post on Zulip Elliot Silver (Jan 13 2022 at 20:50):

Jens Villadsen said:

Before you type anything more ... a wild guess would be that the requirements by HL7 org is beyond affiliate needs

Or a non-affiliate.

view this post on Zulip Grahame Grieve (Jan 13 2022 at 20:50):

what I don't know how to automate is:

  • check that the proposed release is administratively correct (e.g. business rules)
  • check that the publication succeeded (there is a myriad of things go wrong in practice, including file locking problems)

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 20:51):

Grahame Grieve said:

what does it automate?

roughly:

  1. setup the folders as needed
  2. clone what is needed - ig-history and ig-registry
  3. create the publish.ini if one does not exist
  4. run a first build of the ig
  5. create/update the package.ini
  6. run the final publish step
  7. do a minor update for the history page template to work

view this post on Zulip Grahame Grieve (Jan 13 2022 at 20:51):

everything else is already automated

view this post on Zulip Grahame Grieve (Jan 13 2022 at 20:51):

what's the package.ini?

view this post on Zulip Grahame Grieve (Jan 13 2022 at 20:51):

and what's the minor update?

view this post on Zulip Jens Villadsen (Jan 13 2022 at 20:52):

@Jose Costa Teixeira I just found the first bug ... you don't start deleting any existing www-folder

view this post on Zulip Grahame Grieve (Jan 13 2022 at 20:53):

the main change I want to make to the existing -go-publish process is that the release information is buried in a local edited copy of the package-list.json file, and that's massively confusing for editors who think that they're somehow maintaining package-list.json in their repository, and keep trying to revise the past publication history, but they can't

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 20:53):

(trying to catch up here)

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 20:53):

Grahame Grieve said:

the main change I want to make to the existing -go-publish process is that the release information is buried in a local edited copy of the package-list.json file, and that's massively confusing for editors who think that they're somehow maintaining package-list.json in their repository, and keep trying to revise the past publication history, but they can't

this is good to know

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 20:54):

Grahame Grieve said:

and what's the minor update?

IIRC it's about applying a skin to the history page

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 20:56):

Jens Villadsen said:

Jose Costa Teixeira I just found the first bug ... you don't start deleting any existing www-folder

possibly. I don't think that causes anything to be broken. But once this is working fine we can add this

view this post on Zulip Jens Villadsen (Jan 13 2022 at 20:56):

it caused my stuff to break

view this post on Zulip Jens Villadsen (Jan 13 2022 at 20:57):

because the script reads any exisitng file BEFORE it writes it

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 21:02):

ah, you already had a publish.ini and that messed up the next build?
If so, since the publish.ini is input, that is exactly what you should expect - broken input, broken output (and I'm not avoiding the issue, just challenging my own assumption)

view this post on Zulip Grahame Grieve (Jan 13 2022 at 21:04):

@Elliot Silver :
https://confluence.hl7.org/display/FHIR/NPM+Package+Specification#NPMPackageSpecification-ImplementationGuideCanonical + I updated the Package List doco to point ot that

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 21:06):

about the publish.ini:
should the publish.ini url be http://hl7.dk/fhir/dk-core ? that is what I am getting (what the .py script is initializing it with)

view this post on Zulip Jens Villadsen (Jan 13 2022 at 21:10):

it should - by suggestion from @Grahame Grieve - be http://hl7.dk/fhir

view this post on Zulip Jens Villadsen (Jan 13 2022 at 21:10):

which also sounds correct to me

view this post on Zulip Jens Villadsen (Jan 13 2022 at 21:10):

and that value should probably be taken from a new field in the release.json

view this post on Zulip Jens Villadsen (Jan 13 2022 at 21:11):

Jose Costa Teixeira said:

ah, you already had a publish.ini and that messed up the next build?
If so, since the publish.ini is input, that is exactly what you should expect - broken input, broken output (and I'm not avoiding the issue, just challenging my own assumption)

I had the www folder from a previous run which failed

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 21:13):

Jens Villadsen said:

Jose Costa Teixeira said:

ah, you already had a publish.ini and that messed up the next build?
If so, since the publish.ini is input, that is exactly what you should expect - broken input, broken output (and I'm not avoiding the issue, just challenging my own assumption)

I had the www folder from a previous run which failed

you can try to see if you don't mind - replace ig_canonical with base in the py script, in both lines 238 and 254

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 21:13):

Jens Villadsen said:

and that value should probably be taken from a new field in the release.json

that can be done indeed

view this post on Zulip Jens Villadsen (Jan 13 2022 at 21:14):

another piece of feedback - if you have a evaluation statement and stop the program due to something that is not alligned - do print out what is not alligned for debugging purposes

view this post on Zulip Jens Villadsen (Jan 13 2022 at 21:15):

and then you can exit

view this post on Zulip Grahame Grieve (Jan 13 2022 at 21:15):

should the publish.ini url be http://hl7.dk/fhir/dk-core

where is the folder published?

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 21:16):

@Grahame Grieve I am moving towards the go-publish and if I need to trim the release script (or remove it completely), i'll do it. So I'm trying to follow the new process.
I'm currently stuck at the fact that it tells me it expects the hl7.org/<country>/<...>
https://chat.fhir.org/#narrow/stream/179252-IG-creation/topic/IG.20Release.20Publication.20procedure/near/267770291

view this post on Zulip Grahame Grieve (Jan 13 2022 at 21:16):

if the publish.ini has a url of {url} then the browser should be able to find {url}/publish.ini e.g.

view this post on Zulip Jens Villadsen (Jan 13 2022 at 21:16):

http://hl7.dk/fhir/ is the root

view this post on Zulip Grahame Grieve (Jan 13 2022 at 21:16):

http://hl7.org/fhir/publish.ini

view this post on Zulip Grahame Grieve (Jan 13 2022 at 21:16):

huh the HL7.org webserver won't serve that file but it does exist.

view this post on Zulip Grahame Grieve (Jan 13 2022 at 21:17):

https://fhir.org/guides/publish.ini does get served

view this post on Zulip Jens Villadsen (Jan 13 2022 at 21:17):

http://hl7.dk/fhir/dk-core is none-existing as it is named http://hl7.dk/fhir/core

view this post on Zulip Grahame Grieve (Jan 13 2022 at 21:18):

@Jose Costa Teixeira I guess it's this code:

if (!check(res, p.length == 4 && "hl7".equals(p[0]) && "fhir".equals(p[1]), "Package Id '"+id+"' is not valid:  must have 4 parts (hl7.fhir.[realm].[code]")) {
        return res;
      }
      realm = p[2];
      code = p[3];
      if (!check(res, canonical != null && (canonical.equals("http://hl7.org/fhir/"+realm+"/"+code)  || canonical.equals("http://hl7.org/fhir/smart-app-launch")), "canonical URL of "+canonical+" does not match the required canonical of http://hl7.org/fhir/"+realm+"/"+code)) {
        return res;
      }
      if (!check(res, canonical.startsWith(url), "Proposed canonical '"+canonical+"' does not match the web site URL '"+url+"'")) {
        return res;
      }

view this post on Zulip Grahame Grieve (Jan 13 2022 at 21:19):

why don't those rules apply?

view this post on Zulip Jens Villadsen (Jan 13 2022 at 21:20):

### Creating the web root folder
Traceback (most recent call last):
  File "release.py", line 224, in <module>
    os.makedirs(os.path.join(webrootfolder), exist_ok=True)
TypeError: makedirs() got an unexpected keyword argument 'exist_ok'

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 21:21):

@Jens Villadsen that is a python version thing, I think. let me get to that

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 21:25):

Grahame Grieve said:

Jose Costa Teixeira I guess it's this code:

if (!check(res, p.length == 4 && "hl7".equals(p[0]) && "fhir".equals(p[1]), "Package Id '"+id+"' is not valid:  must have 4 parts (hl7.fhir.[realm].[code]")) {
        return res;
      }
      realm = p[2];
      code = p[3];
      if (!check(res, canonical != null && (canonical.equals("http://hl7.org/fhir/"+realm+"/"+code)  || canonical.equals("http://hl7.org/fhir/smart-app-launch")), "canonical URL of "+canonical+" does not match the required canonical of http://hl7.org/fhir/"+realm+"/"+code)) {
        return res;
      }
      if (!check(res, canonical.startsWith(url), "Proposed canonical '"+canonical+"' does not match the web site URL '"+url+"'")) {
        return res;
      }

Yes. This says that if the package name is hl7.fhir.xxx.yyy, the canonical must have the form hl7.org/fhir/xx/yyy

view this post on Zulip Grahame Grieve (Jan 13 2022 at 21:25):

indeed. Why do you think that shouldn't apply?

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 21:26):

from past discussions I understood that a affiliate package would be hl7.fhir.country.whatever

view this post on Zulip Grahame Grieve (Jan 13 2022 at 21:26):

yes that is the case.

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 21:27):

but some Belgian packages would be hosted in hl7belgium.org/whatever and some other packages in another url

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 21:27):

same thing with the danish

view this post on Zulip Grahame Grieve (Jan 13 2022 at 21:28):

then I'll accept a request to that nature from the affiliates and update the code accordingly

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 21:29):

these urls are getting a bit clearer to me, which is good.

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 21:30):

a request from the affiliates: is this chat with a few upvotes ok?

view this post on Zulip Jens Villadsen (Jan 13 2022 at 21:31):

The danish would be hosted under hl7.dk

view this post on Zulip Grahame Grieve (Jan 13 2022 at 21:32):

no. An email from an affiliate technical representative cc:ed to the affiliate chair will do the job

view this post on Zulip Jens Villadsen (Jan 13 2022 at 21:32):

As in eg. https://hl7.dk/fhir/core/1.1.0/

view this post on Zulip Jens Villadsen (Jan 13 2022 at 21:33):

@Grahame Grieve so I need to send a mail to myself?

view this post on Zulip Grahame Grieve (Jan 13 2022 at 21:34):

no you can just send me the email directly!

view this post on Zulip Chris Moesel (Jan 13 2022 at 21:34):

Jens Villadsen said:

for starters - the fsh configuration page

This one's mine. I can clarify on that page.

view this post on Zulip Jens Villadsen (Jan 13 2022 at 21:37):

Speaking of bugs: @Grahame Grieve could you have a look at https://github.com/hapifhir/org.hl7.fhir.core/issues/703

view this post on Zulip Jens Villadsen (Jan 13 2022 at 21:39):

And regarding

no. An email from an affiliate technical representative cc:ed to the affiliate chair will do the job

I think that should be shared in the IC forum as well

view this post on Zulip Jens Villadsen (Jan 13 2022 at 21:40):

As well as eg. HL7 EU

view this post on Zulip Grahame Grieve (Jan 13 2022 at 21:41):

what forum?

view this post on Zulip Jens Villadsen (Jan 13 2022 at 21:41):

International Council

view this post on Zulip Grahame Grieve (Jan 13 2022 at 21:42):

the email list?

view this post on Zulip Jens Villadsen (Jan 13 2022 at 21:43):

Haha .... We also meet from time to time

view this post on Zulip Jens Villadsen (Jan 13 2022 at 21:43):

So we are a bit more than an email list

view this post on Zulip Jens Villadsen (Jan 13 2022 at 21:44):

https://confluence.hl7.org/display/IC/International+Council

view this post on Zulip Grahame Grieve (Jan 13 2022 at 21:44):

is the IC meeting by zoom this week? I would usually do an in person update but no one has spoken to me about that

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 21:48):

I thik it's on the 26th?

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 21:48):

https://confluence.hl7.org/display/IC/January+26th%2C+2022+International+Council+Virtual+Meeting+Agenda

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 21:49):

I think an in person update would make sense. We can notify Line to add to the agenda

view this post on Zulip Grahame Grieve (Jan 13 2022 at 21:50):

I might be able to do it on the 26th - depends on the timezone. I'm off in the bush for most of the WGM, but I get back late that day

view this post on Zulip Grahame Grieve (Jan 13 2022 at 22:19):

ok https://github.com/HL7/fhir-ig-publisher/pull/369/files#diff-ccc6b4e460052b5fd03d8984f50b8d8b82bc0be544295b962203c8c622bd16a1

view this post on Zulip Grahame Grieve (Jan 13 2022 at 22:19):

should be released in the next 24 hours

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 22:52):

Does this still support IHE's profiles.ihe.net?

view this post on Zulip Jose Costa Teixeira (Jan 13 2022 at 22:54):

IHE's packages are not in the form hl7.fhir.... but I don't see if the logic supports other URLs if the package name is different

view this post on Zulip Grahame Grieve (Jan 13 2022 at 22:57):

at present, it doesn't check IHE canonicals at all. if someone wants to tell me what the IHE rules for packages and canonicals I can add QA checks for IHE too

view this post on Zulip Grahame Grieve (Jan 13 2022 at 22:57):

I don't need to check IHE - it's just QA for IHE if I do

view this post on Zulip Oliver Egger (Jan 14 2022 at 08:23):

for IHE the web publication is at https://profiles.ihe.net/ e.g. IG with package name IHE.ITI.PIXm#3.0.0 lives at https://profiles.ihe.net/ITI/PIXm/index.html, canonical http://profiles.ihe.net/ITI/PIXm

@John Moehrke can you pleas confirm also?

view this post on Zulip Jens Villadsen (Jan 14 2022 at 10:35):

@Jose Costa Teixeira

you can try to see if you don't mind - replace ig_canonical with base in the py script, in both lines 238 and 254

Looks like it fixes the most troublesome parts. Will you fix this in your 'normative' repository - wherever that may be?

view this post on Zulip Jose Costa Teixeira (Jan 14 2022 at 10:38):

Yes, I will commit that change

view this post on Zulip Jose Costa Teixeira (Jan 14 2022 at 10:38):

there's no normative repository but there may be a primary source, after we have this process clear.

view this post on Zulip Jens Villadsen (Jan 14 2022 at 10:40):

where's the 'temporary repository' then?

view this post on Zulip Jose Costa Teixeira (Jan 14 2022 at 10:40):

My plan is (please let me know if it makes sense)

  1. Get something to work (done) that can be used in a CI process and document it
  2. Adapt this to the new approach that Grahame added (including the go-publish thing)
  3. If there is a script to facilitate the work, I'll put it in a repo somewhere where it can be reused (github does have reusable workflows since recently, I think)

view this post on Zulip Jose Costa Teixeira (Jan 14 2022 at 10:41):

Jens Villadsen said:

where's the 'temporary repository' then?

https://github.com/costateixeira/testrel/blob/master/release.py for now

view this post on Zulip Jose Costa Teixeira (Jan 14 2022 at 10:42):

I am making the script to show that there is value in some automation. If this has value for HL7 then I hope we can put a "community version" somewhere

view this post on Zulip Jose Costa Teixeira (Jan 14 2022 at 10:43):

if not, then it will be a bottom-up volunteer initiative :)

view this post on Zulip Jens Villadsen (Jan 14 2022 at 11:02):

Or make this the priority of the new CSDO @Daniel Vreeman

view this post on Zulip John Moehrke (Jan 14 2022 at 13:08):

Oliver Egger said:

for IHE the web publication is at https://profiles.ihe.net/ e.g. IG with package name IHE.ITI.PIXm#3.0.0 lives at https://profiles.ihe.net/ITI/PIXm/index.html, canonical http://profiles.ihe.net/ITI/PIXm

John Moehrke can you pleas confirm also?

yes, current thinking for IHE is

https://profiles.ihe.net/${domain}/${profile}

I have adjusted the guidance to lowercase the IG id, while we continue to use uppercase in the canonical.

current thinking. Glad to be guided.

view this post on Zulip Jose Costa Teixeira (Jan 14 2022 at 18:33):

John Moehrke said:

https://profiles.ihe.net/${domain}/${profile}

I have adjusted the guidance to lowercase the IG id, while we continue to use uppercase in the canonical.

Do you mean PIXm in the canonical, but pixm in the id?

view this post on Zulip John Moehrke (Jan 14 2022 at 21:14):

yes

view this post on Zulip John Moehrke (Jan 14 2022 at 21:15):

that has not yet been fixed. pull-request has been submitted, ITI committee needs to agree, and publication governance.

view this post on Zulip Vassil Peytchev (Jan 14 2022 at 21:23):

And are there any case requirements for the package name?

view this post on Zulip Jose Costa Teixeira (Jan 14 2022 at 22:20):

I think that was discussed - yes, package name is expected to be lower case (I can't find the spec but this for example describes the rules: https://www.npmjs.com/package/validate-npm-package-name)

view this post on Zulip Jose Costa Teixeira (Jan 14 2022 at 22:25):

@Grahame Grieve a few things as I'm exploring:

      if (!check(res, canonical == null, "canonical URL not found")) {

(publicationprocess, line 136)
should this be canonical != null?

view this post on Zulip Jose Costa Teixeira (Jan 14 2022 at 22:26):

also, line 152:

String destination = tho || cql ? rootFolder : PastProcessHackerUtilities.useRealm(realm, code) ? Utilities.path(rootFolder, realm, code) :  Utilities.path(rootFolder, code);

I'm not sure this is compatible with the scenario for BE (or others). It seems to require that the folder structure will be web-root\be\core.
However, there is no bein the URL for the publication: it's hl7belgium.org/profiles/fhir/core


Last updated: Apr 12 2022 at 19:14 UTC