Stream: IG creation
Topic: Templates not found during cloud build?
Kevin Power (Jan 31 2022 at 17:46):
Seems at least some builds are failing due to not being able to find the templates (again):
http://build.fhir.org/ig/HL7/genomics-reporting/branches/master/failure/build.log
Load Template from hl7.fhir.template#current (00:40.0878)
org.hl7.fhir.exceptions.FHIRException: The package 'hl7.fhir.template' has no entry on the current build server
Kevin Power (Jan 31 2022 at 17:47):
Another build failing due to the same issue:
http://build.fhir.org/ig/HL7/fhir-registry-protocols-ig/branches/master/failure/build.log
Kevin Power (Jan 31 2022 at 17:48):
I was able to build the 'genomics-reporting' IG locally without an error.
Kevin Power (Jan 31 2022 at 17:51):
@Josh Mandel - Any chance this is related to your change to clean up the build server?
Elliot Silver (Jan 31 2022 at 17:51):
If I recall, several of the templates have never had an official release, and aren't actually found anywhere but the build server.
Lloyd McKenzie (Jan 31 2022 at 17:54):
@Grahame Grieve - I thought you'd created official releases of these?
Lloyd McKenzie (Jan 31 2022 at 18:04):
Would this be related to the purge of old CI-builds? (If so, we definitely need to stop that from happening - some template CI builds might not change for years, but the templates still need to work...)
David Pyke (Jan 31 2022 at 18:07):
Yep, my carequality template is broken (but works as a github URL)
Lloyd McKenzie (Jan 31 2022 at 18:10):
You can use the https://fhir.github.io/auto-ig-builder/builds.html page to force a re-build of any template that's giving you grief as a short-term solution.
David Pyke (Jan 31 2022 at 18:10):
Yep, I had to push a trivial change as I hadn't rebuilt it in over a year
Lloyd McKenzie (Jan 31 2022 at 18:10):
The build page lets you re-trigger a build without a new commit.
David Pyke (Jan 31 2022 at 18:11):
Yeah but it only goes back a few months, my last template build was 1.5 years ago
Kevin Power (Jan 31 2022 at 18:34):
Yea, I don't see https://github.com/HL7/ig-template-fhir on the 'auto-ig-builder' page - and I think that is the one the genomics-reporting IG is failing on.
David Pyke (Jan 31 2022 at 18:40):
If it's urgent, you can change the template line in ig.ini to the github repository. That will let it build until the template is available again. If it's not urgent, I wouldn't bother
John Moehrke (Jan 31 2022 at 18:52):
the auto-ig-builder didn't show everything. no idea why I had IGs that I wanted to rebuild but were not found on the auto-ig-builder. (e.g. FormatCode)
Grahame Grieve (Jan 31 2022 at 19:57):
@Lloyd McKenzie I released all of them I know about, but you actually have to point at the release not current.
Grahame Grieve (Jan 31 2022 at 19:57):
if an IG is scrubbed because it gets to old, it also falls out of the auto-build summary page
Grahame Grieve (Jan 31 2022 at 19:57):
so you can't rebuild it from there - have to make a trivial commit.
Grahame Grieve (Jan 31 2022 at 19:58):
but there is a series of things broken from @Josh Mandel cleaning out old builds
Josh Mandel (Jan 31 2022 at 19:59):
You can always follow https://github.com/fhir/auto-ig-builder#if-you-want-to-manually-trigger-a-build to manually trigger a build.
Josh Mandel (Jan 31 2022 at 19:59):
Re: new problems from clearing out builds: yeah, let me know if there's anything that requires an immediate workaround. In the long run, this process helps us avoid dependencies on infra that we intend to be ephemeral.
Lloyd McKenzie (Jan 31 2022 at 20:02):
We don't want to clear template builds ever. 'current' needs to always be available.
Lloyd McKenzie (Jan 31 2022 at 20:03):
And the same is true of anything HL7 has published. We'll have to figure out a mechanism for retaining things others have published where they have a similar need.
Grahame Grieve (Jan 31 2022 at 20:05):
this is all solved by the master/main issue
Grahame Grieve (Jan 31 2022 at 20:05):
but also, Josh is right - people should not be using current templates as a matter of course
Josh Mandel (Jan 31 2022 at 20:22):
'current' needs to always be available
I think the package server is handling this now, without depending on content staying on the CI server. Correct me if this is wrong.
Grahame Grieve (Jan 31 2022 at 20:24):
not if people point to current - that uses build.fhir.org
Josh Mandel (Jan 31 2022 at 20:30):
Didn't we come up with a solution where even "current" would reference a package in the package server?
Josh Mandel (Jan 31 2022 at 20:30):
Maybe we did and it still needs to be built, or maybe I'm misremembering where we landed.
Grahame Grieve (Jan 31 2022 at 20:31):
we specced it out, but we didn't implement it yet.
Josh Mandel (Jan 31 2022 at 20:31):
Cool. So a near-term solution where people keep branches alive as-needed isn't crazy/unmanageable.
Grahame Grieve (Jan 31 2022 at 20:34):
but we are going to keep master/main indefinitely, right?
Josh Mandel (Jan 31 2022 at 20:37):
That has been suggested. I'd rather not, in general, given how many short-lived projects use the auto-build infra. But maybe for packages that have an official publication associated with them, this would make more sense.
Grahame Grieve (Jan 31 2022 at 20:39):
can we do it for projects in fhir and hl7 orgs? they always have some approval associated with them
Josh Mandel (Jan 31 2022 at 20:42):
Sure.
Josh Mandel (Jan 31 2022 at 21:44):
OK, so I've set up # days and the exception orgs/branches for more flexibility at https://github.com/FHIR/auto-ig-builder/commit/61f72ff0f8f1623658d100cbd6c9f350637a1494
Josh Mandel (Jan 31 2022 at 21:45):
Should be deployed now...
Josh Mandel (Jan 31 2022 at 21:46):
Will skip auto-clean on branches that are in HL7 or FHIR org and have a branch name of master or main
Grahame Grieve (Jan 31 2022 at 21:47):
thanks
Nathan Davis (Jan 31 2022 at 22:15):
I'm a bit lost in this conversation. Can we use #current for our template? if not, what should I use? right now I have template = hl7.fhir.template#current
Gino Canessa (Jan 31 2022 at 22:17):
I was going to bring up branches named R4B, but it will be published before this becomes an issue, right? :angel:
Grahame Grieve (Jan 31 2022 at 22:20):
yes
Grahame Grieve (Jan 31 2022 at 22:23):
you can use #current, but you'll be at the mercy of whatever happens in the build system
John Moehrke (Jan 31 2022 at 22:24):
please do the same with IHE as you did with HL7
John Moehrke (Jan 31 2022 at 22:25):
Generally speaking IHE had not switched over to a versioned template as it was not clear that we had progressed into needing that. So, I presume that is ready to go now?
John Moehrke (Jan 31 2022 at 22:31):
We need to have somewhere where the version of the templates is easily findable. I preferred to use @current because I didn't have a specific need for a locked in version and didn't want to be revising my IG simply because the template changed.
Josh Mandel (Jan 31 2022 at 22:38):
John Moehrke: please do the same with IHE as you did with HL7
Done.
Josh Mandel (Jan 31 2022 at 22:40):
I preferred to use @current because I didn't have a specific need for a locked in version
This is what semver is for: you should be declaring a major version that you know you're compatible with and automatically getting whatever is the latest published version compatible with that.
Grahame Grieve (Jan 31 2022 at 22:41):
I don't think that the template loader works that way. I should probably make it so. but if you don't nominate any version, then you just get the latest release. That's the recommended approach, except that Lloyd keeps saying to use current
Josh Mandel (Jan 31 2022 at 22:42):
Yeah, "latest release vs latest commit on github" seems much safer.
Kevin Power (Jan 31 2022 at 22:43):
Sorry for the dumb question, but where I find the released versions of the templates?
John Moehrke (Jan 31 2022 at 22:45):
sorry for the dumb question also, but what is the difference in calling for latest template release vs calling for current?
Grahame Grieve (Jan 31 2022 at 22:45):
well, you shouldn't need to find anything. The IG publisher will do the finding for you
Grahame Grieve (Jan 31 2022 at 22:45):
but if you really want to know, they are released here: https://fhir.org/templates/
John Moehrke (Jan 31 2022 at 22:46):
if we really must put in a version, then we need to know manually what is the best version.
Grahame Grieve (Jan 31 2022 at 22:46):
what is the difference in calling for latest template release vs calling for current?
current is build automatically from the github source when someone commits something. When it's stable, then someone asks me, and I release a new version on fhir.org/templates, and that's the officially released version
Grahame Grieve (Jan 31 2022 at 22:46):
so just don't put any version - then you get the latest approved release
John Moehrke (Jan 31 2022 at 22:47):
ah, so not saying # something will use the latest released
Kevin Power (Jan 31 2022 at 22:48):
Thanks all
John Moehrke (Jan 31 2022 at 22:48):
@Grahame Grieve please make the latest IHE template the one currently built... I didn't know I needed (or could) do something like that.
John Moehrke (Jan 31 2022 at 22:49):
thus there was no latest IHE, and thus all our builds failed when the ci-build for the IHE template was deleted.
Grahame Grieve (Jan 31 2022 at 22:49):
do I need to release any other templates (@Lloyd McKenzie )
Chris Moesel (Jan 31 2022 at 22:50):
So... I'm noticing that IGs that still use hl7.fhir.template#current are failing right now (at least failing locally for me). It looks like no one triggered a rebuild of the templates so we can get them back onto the build server. Was that intentional? I can try to trigger a build, but I don't want to mess things up if it's not part of the plan.
John Moehrke (Jan 31 2022 at 22:54):
looks like many template authors have kicked an IG build in the past 24 hours... likely they all are candidates for formal release?
Chris Moesel (Jan 31 2022 at 22:54):
In related news, I just tried to get to the docs on how to create IGs using the templated approach (https://build.fhir.org/ig/FHIR/ig-guidance/index.html) and it 404s. Probably the same "old build" issue. (@Lloyd McKenzie)
Lloyd McKenzie (Jan 31 2022 at 23:03):
Yes. Purging all of the old releases without talking about it first was unfortunate :( I've pushed a commit to trigger a re-build.
"current" is for people to have a chance to use the latest content before it manifests in the official build. We're supposed to be moving to a monthly 'release' for the templates, but haven't gotten there yet. Even then, the 'current' version should remain for everything in perpetuity.
The templates I'm aware of are: base, ca, davinci, fhir, gravity, hl7, utg
Grahame Grieve (Jan 31 2022 at 23:53):
@John Moehrke the current release of the IHE template is 0.4.0, and that's the version specified in github. If you want me to release a new version, you need to create an entry in package-list.json with a new version number and a description of the changes in it, and then I'll do the release
John Moehrke (Jan 31 2022 at 23:59):
I think that is the one I wanted. When I looked at the link you gave, I had not seen the IHE one there at that time. I guess I missed it... then my builds should not have failed, because they all are just calling for the template without a any # or calling for 0.4.0 already. so not sure why I was getting failures.
Grahame Grieve (Jan 31 2022 at 23:59):
what's failing then?
Elliot Silver (Feb 01 2022 at 00:02):
Josh Mandel said:
you should be declaring a major version that you know you're compatible with
Is that a semver major version, or an HL7 major version? :stuck_out_tongue_wink:
Grahame Grieve (Feb 01 2022 at 00:02):
@Elliot Silver playing the 'spank me' card
Elliot Silver (Feb 01 2022 at 00:04):
Sorry, I'll go back in my corner now.
John Moehrke (Feb 01 2022 at 00:09):
I had thought the same thing
Josh Mandel (Feb 01 2022 at 00:43):
Yes. Purging all of the old releases without talking about it first was unfortunate :( I've pushed a commit to trigger a re-build.
I'm sorry for the unnecessary frustration this has caused! (To be clear, we talked about it extensively in #committers -- it just took me way too long to get around to implementing the changes we had decided on.)
Josh Mandel (Feb 01 2022 at 00:44):
(There is actually some good discipline in randomly deleting stuff that you claim is ephemeral -- perhaps similar to spending your downtime budget on a service that has been rock solid by intentionally turning it off for a while...)
John Moehrke (Feb 01 2022 at 12:22):
I think it was a useful exercise, with hardly a downside except a bit of head scratching.
Chris Moesel (Feb 01 2022 at 13:27):
I just triggered a rebuild of hl7.fhir.template, so builds that rely on hl7.fhir.template#current should start succeeding again.
No need to apologize, @Josh Mandel. As you said, it was previously discussed, so I don't think there's much more you could have done. Now we all understand the build ecosystem just a little bit better. And thank you for all of the things you do (and have done) for this community!
Oliver Egger (Feb 01 2022 at 15:07):
we even detected that we had a dependency on #current which should not have stayed like this, so glad @Josh Mandel that we could resolve it now and will not have problems later on
David Pyke (Feb 01 2022 at 16:37):
My carequality template is still broken. I've rebuilt it but it can't be found in packages or packages2 and when it tries to fetch it using the github call, it's using the old address (formerly my private but now it's in carequality's)
David Pyke (Feb 01 2022 at 16:38):
Strangely, it works locally, but not on the ci builder
David Pyke (Feb 01 2022 at 16:41):
Link: https://build.fhir.org/ig/Carequality/CEQSubscription/branches/master/failure/build.log
David Pyke (Feb 01 2022 at 16:41):
@Grahame Grieve
John Moehrke (Feb 01 2022 at 16:46):
David are you calling for by version, by #current, or bare?
David Pyke (Feb 01 2022 at 16:48):
Bare. I'm re-building the template, just in case
John Moehrke (Feb 01 2022 at 16:50):
and is your template relying on predicate template by version, by #current, or bare?
David Pyke (Feb 01 2022 at 16:50):
There's only a current version so, that's all there is
John Moehrke (Feb 01 2022 at 16:51):
so then go to not specifying #current; and you will then get the latest
David Pyke (Feb 01 2022 at 16:52):
That's what I've done
John Moehrke (Feb 01 2022 at 16:52):
oh, drat.
David Pyke (Feb 01 2022 at 16:52):
Still fails saying "Failed to determine latest version of package fhir.carequality.template from server: http://packages.fhir.org"
David Pyke (Feb 01 2022 at 16:53):
I'm assuming some update of packages needs to be done?
David Pyke (Feb 01 2022 at 16:55):
From the log, it seems that the template is gone from packages and packages2
http://packages.fhir.org (Package not found: fhir.carequality.template)
https://packages2.fhir.org/packages (Package not found: fhir.carequality.template)
John Moehrke (Feb 01 2022 at 16:58):
well there is no formally published fhir.carequality.template... Grahame indicated that you must ask him to do that to get it into http://fhir.org/templates/
David Pyke (Feb 01 2022 at 16:58):
It worked before the great purge of 2022
John Moehrke (Feb 01 2022 at 16:58):
but, I understood that using latest would find it in ci-build
John Moehrke (Feb 01 2022 at 16:59):
so might be that your ci-build of the carequality template is failing.. even though it says success
John Moehrke (Feb 01 2022 at 16:59):
no idea how to debug that... @Jose Costa Teixeira ?
Jose Costa Teixeira (Feb 01 2022 at 17:00):
i've just gone through something that can be similar
Jose Costa Teixeira (Feb 01 2022 at 17:00):
@David Pyke where is the source?
David Pyke (Feb 01 2022 at 17:01):
https://github.com/Carequality/ig-template-carequality
Jose Costa Teixeira (Feb 01 2022 at 17:07):
- is this template an authorized one? (not sure this is essential)
Jose Costa Teixeira (Feb 01 2022 at 17:08):
- was this template built using the webhook?
Jose Costa Teixeira (Feb 01 2022 at 17:11):
this template actually doesn't have scripts so it should be fine
Jose Costa Teixeira (Feb 01 2022 at 17:11):
@David Pyke did you setup the webhook on the template repo?
David Pyke (Feb 01 2022 at 17:12):
Yep, it's building. You can find it on the build page
Jose Costa Teixeira (Feb 01 2022 at 17:31):
https://github.com/Carequality/ig-template-carequality/blob/master/package-list.json#L13
Jose Costa Teixeira (Feb 01 2022 at 17:31):
comma missing
John Moehrke (Feb 01 2022 at 17:33):
got to love json
Lloyd McKenzie (Feb 01 2022 at 17:34):
XML Rules!
David Pyke (Feb 01 2022 at 17:35):
Yay! The exciting thing is that it worked before.
John Moehrke (Feb 01 2022 at 17:36):
yup, as I said... love json
John Moehrke (Feb 01 2022 at 17:38):
this is why we are all humbled and willing to lend a peer our eyes when things don't work right. Thanks @Jose Costa Teixeira
David Pyke (Feb 01 2022 at 17:58):
Nope, that didn't fix the problem.
David Pyke (Feb 01 2022 at 17:58):
I have a feeling that packages or packages2 needs to be manually updated. Who do I bother about that?
Lloyd McKenzie (Feb 01 2022 at 18:04):
@Grahame Grieve
Lloyd McKenzie (Feb 01 2022 at 18:04):
Or maybe @Mark Iantorno?
Jose Costa Teixeira (Feb 01 2022 at 18:06):
what is the issue?
David Pyke (Feb 01 2022 at 18:06):
Same issue, the template is not found in packages and packages2. It used to be
Jose Costa Teixeira (Feb 01 2022 at 18:07):
can you add #current to your ig.ini?
Jose Costa Teixeira (Feb 01 2022 at 18:07):
in the template , i mean
Jose Costa Teixeira (Feb 01 2022 at 18:07):
it does seem to work on my machine
John Moehrke (Feb 01 2022 at 18:08):
local is working... not ci-build
David Pyke (Feb 01 2022 at 18:08):
I can try it with #current on the template line and see if the ci-build works
David Pyke (Feb 01 2022 at 18:11):
Yep, if I force #current, it works. I'll stick with that
John Moehrke (Feb 01 2022 at 18:12):
that was kinda what I was asking about... a version or nothing means to pull from the formal template publication place. where as #current says pull from an ci-build
David Pyke (Feb 01 2022 at 18:13):
Ah, you'd think that if there was no formal publication version it would fall back.
John Moehrke (Feb 01 2022 at 18:15):
no, I expected blank, or declared version would need to come from the formal publication that Grahame is the gatekeeper, and carequality was not already there.
John Moehrke (Feb 01 2022 at 18:15):
your experience confirms that
John Moehrke (Feb 01 2022 at 18:16):
so #current should be unusual, but you are... unusual
Grahame Grieve (Feb 01 2022 at 19:15):
it's my choice that not specifying a version doesn't fall back to the current build - you have to specify that explicitly. And this template was never released so it never appeared on either packages server. It looks to me like there was a transient problem created by the purge because you had two different repositories claiming to be the same template, and the one that wasn't built was winning (the algorithm for choosing which one is non-determinate).
Yolanda (Feb 01 2022 at 19:44):
I am trying to catch up with such a long discussion. I tried to build DTR IG template line "with" and "without" #current, they both failed. Here is the log: https://build.fhir.org/ig/HL7/davinci-dtr/branches/master/failure/build.log Here is the source code: https://github.com/HL7/davinci-dtr/tree/master Any thoughts? Thanks
Yolanda (Feb 01 2022 at 19:44):
@Lloyd McKenzie
Lloyd McKenzie (Feb 01 2022 at 19:56):
@Grahame Grieve - can you check that the Da Vinci template is in the package repository?
Grahame Grieve (Feb 01 2022 at 19:56):
it is
Grahame Grieve (Feb 01 2022 at 19:56):
Lloyd McKenzie (Feb 01 2022 at 20:01):
@Yolanda, I'm not seeing the build log there anymore. Does that mean it's now built successfully?
Grahame Grieve (Feb 01 2022 at 20:06):
works for me
Yolanda (Feb 01 2022 at 21:11):
Yes, it seems like it resolved itself. Thanks Lloyd and Grahame for checking it.
Ted Klein (Feb 02 2022 at 22:07):
so I am late to this cluster** party, but you guys threw out another baby with the bath water. THO/UTG git branch builds that are still in Consensus Review or QA valuation for the community in the UTG workflow got dumped as well. We DO NOT EVER want to remove those branch builds if the content is still in voting consensus review. I also don't see an easy way to trigger a rebuild of a branch without making a trivial change (eg adding whitespace to <label> value ) with a commit and push - which if there is change to content due to updates to the IG Publisher means that folks seeing it now may see different content to what earlier voters saw and voted on. The policy for THO consensus review is that the content is locked while the vote is open. But cleaning the branch build and rebuilding with a different template, publisher, etc. may give a change which may technically invalidate the voting so far. Doing this to clear up the UTG builds was very shortsighted.
Grahame Grieve (Feb 02 2022 at 22:14):
well, you have no choice right now but to make a whitespace commit
Grahame Grieve (Feb 02 2022 at 22:14):
how long is voting consensus review?
Ted Klein (Feb 02 2022 at 22:16):
TSMG and TSC have to figure out the timeframe; they pulled back the changes I recommended for timeouts. Right now it is unlimited. One ticket has been out for voting for six months.
Ted Klein (Feb 02 2022 at 22:17):
however items that are sent to Submit and then back and forth to QA and back may be a long time as well. Thank heavens I have all the stuff in Draft in Bitbucket so it is in a private place for UTG.
Ted Klein (Feb 02 2022 at 22:18):
this is another motivator for a governance decision on timeouts. but many proposals will be not less than 90 days. cleaning branches out so things are wiped when supposed to be locked is a Real Bad Thing.
Grahame Grieve (Feb 02 2022 at 22:19):
well, how would an algorithm now when something is safe to delete? Given the size of THO, we can't just let them accumulate...
Ted Klein (Feb 02 2022 at 22:23):
Agreed. easy to decide such an algorithm...but the operands are sitting in Jira, not Git...lemme noodle on it...luckily only two got wiped, but if/when UTG/THO maintenance accelerates we may likely have dozens in voting review at any given time. When merged into THO ci build then would be the time to remove the branch build, but I dunno if web hook arguments would be able to indicate what was just merged...
Ted Klein (Feb 02 2022 at 22:24):
could make it a cleanup taks for the curator on merge of approved proposal...couple button clicks only...
Ted Klein (Feb 02 2022 at 22:26):
I may have a solution...lemme do a bit of research...
Josh Mandel (Feb 02 2022 at 22:47):
https://github.com/FHIR/auto-ig-builder#if-you-want-to-manually-trigger-a-build provides instructions for triggering a rebuild without having to make any new commits on your content
Ted Klein (Feb 02 2022 at 22:59):
thanks Josh - I'll check it out. I may have to go over some webhook stuff with you I have in mind...
Ted Klein (Feb 03 2022 at 00:28):
Josh, it might be entertaining - when something very old (scores or hundreds of commits in the past) is the one to be rebuilt, then there might be quite the scrolling operations through the webhooks recent commits list on Git...and when they say 'recent'...will it go back six to ten months?
Josh Mandel (Feb 03 2022 at 01:37):
I would recommend the command line invocation using curl :-)
Ted Klein (Feb 03 2022 at 15:10):
ok. for cleaning up the built branches in UTG - if you can give some kind of a command line or other control, we can just put it into the curator workflow (and maybe trigger it from Jira) when the proposal comes out of Consensus Review and gets integrated into the ci build. This will prevent them from growing indefinitely. If I can wire it into the Jira workflow logic, then it will happen as part of processing and thus cannot be accidentally skipped.
Elliot Silver (Feb 15 2022 at 03:55):
@Lloyd McKenzie , @Grahame Grieve, can we kick off a build of the HL7 Canada template (https://github.com/HL7/ig-template-ca)? Looks to have been lost in the great reclamation.
Grahame Grieve (Feb 15 2022 at 04:00):
You can
Elliot Silver (Feb 15 2022 at 04:08):
Hey, look at that -- I can. I figured I needed to be able to see the webhooks settings page, but @Josh Mandel 's curl command doesn't care whether you are repo admin.
Last updated: Apr 12 2022 at 19:14 UTC
