Stream: IG creation
Topic: branch support in toolchain
Jens Villadsen (Mar 24 2020 at 23:51):
The https://github.com/FHIR/auto-ig-builder states that branches are supported. Yet, I can't see any other branch than master on http://build.fhir.org/ig/hl7dk/dk-medcom-core-r4/branches/ - is that a bug?
Josh Mandel (Mar 24 2020 at 23:56):
Seems like a but, let's see.
Josh Mandel (Mar 24 2020 at 23:57):
To clarify, I don't see any successful builds on any branch -- is this a working IG?
Josh Mandel (Mar 24 2020 at 23:57):
Content at https://github.com/hl7dk/dk-medcom-core-r4 appears missing.
Jens Villadsen (Mar 24 2020 at 23:58):
look at https://github.com/hl7dk/dk-medcom-core-r4/tree/feature/ig-compatible
Jens Villadsen (Mar 24 2020 at 23:58):
that branch contains a bug in the igpublisher that I need to flag to @Grahame Grieve
Josh Mandel (Mar 24 2020 at 23:59):
Thanks, that helps -- I'm not sure what the issue is, but let's revisit when the IG publisher is back up.
Jens Villadsen (Mar 24 2020 at 23:59):
but in order to do so .... @Josh Mandel I need the toolchain to work on branches
Jens Villadsen (Mar 24 2020 at 23:59):
is it down?
Josh Mandel (Mar 25 2020 at 00:00):
No builds are currently working (See #IG creation > Error: unable to resolve package id hl7.fhir.pubpack )
Jens Villadsen (Mar 25 2020 at 00:01):
k
Jens Villadsen (Mar 25 2020 at 00:01):
Josh Mandel (Mar 25 2020 at 00:03):
In the meantime, can I ask you to give me admin permissions on this repo for a few day? I'd like to debug webhook settings, etc.
Jens Villadsen (Mar 25 2020 at 00:04):
as long as you only fiddle with webhooks
Josh Mandel (Mar 25 2020 at 00:04):
You got it.
Jens Villadsen (Mar 25 2020 at 00:05):
done
Josh Mandel (Mar 25 2020 at 00:07):
Perfect. And OK, I see the issue: / in the branch names isn't supported right now. I can fix, but if you want something right away, just rename the branch to feature- instead of feature/
Josh Mandel (Mar 25 2020 at 00:15):
(The issue is that branches feed into file system layout on the build.fhir.org side, so the "best case" fix would be mapping / to - or something during the publication process; this creates confusion because it means branch names aren't transparently converted to URLs... so I'm inclined to just prohibit branches with / from the auto-build... but open to suggestions.)
Jens Villadsen (Mar 25 2020 at 06:03):
That is okay with me
Jens Villadsen (Mar 25 2020 at 06:03):
Just be sure to document it somewhere
Jens Villadsen (Mar 25 2020 at 06:05):
What about the other branches?
Jens Villadsen (Mar 25 2020 at 09:46):
@Josh Mandel - branch is now deleted
Jens Villadsen (Mar 25 2020 at 13:00):
and nothing works .... :worried:
Josh Mandel (Mar 25 2020 at 14:05):
So IGs are building again now in general, so that's a start :-)
Josh Mandel (Mar 25 2020 at 14:06):
Looking at https://github.com/hl7dk/dk-medcom-core-r4/tree/version_0.8 -- I expect this is the only branch that might work, yes?
Josh Mandel (Mar 25 2020 at 14:06):
http://build.fhir.org/ig/hl7dk/dk-medcom-core-r4/branches/ shows the logs
Josh Mandel (Mar 25 2020 at 14:07):
(I re-sent the last github webhook notification just to make sure I could watch all the steps happening.)
Josh Mandel (Mar 25 2020 at 14:08):
Those logs are currently showing a reason for failure that @Grahame Grieve might understand, but @Jens Villadsen should I assume this branch builds for you locally?
java.lang.IndexOutOfBoundsException: Index 0 out of bounds for length 0
at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64)
at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70)
at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:248)
at java.base/java.util.Objects.checkIndex(Objects.java:373)
at java.base/java.util.ArrayList.get(ArrayList.java:426)
at org.hl7.fhir.r5.conformance.ProfileUtilities.getComparer(ProfileUtilities.java:4591)
at org.hl7.fhir.r5.conformance.ProfileUtilities.sortElements(ProfileUtilities.java:4577)
at org.hl7.fhir.r5.conformance.ProfileUtilities.sortElements(ProfileUtilities.java:4579)
at org.hl7.fhir.r5.conformance.ProfileUtilities.sortDifferential(ProfileUtilities.java:4516)
at org.hl7.fhir.igtools.publisher.Publisher.generateSnapshot(Publisher.java:3934)
at org.hl7.fhir.igtools.publisher.Publisher.generateSnapshots(Publisher.java:3902)
at org.hl7.fhir.igtools.publisher.Publisher.loadConformance(Publisher.java:3335)
at org.hl7.fhir.igtools.publisher.Publisher.createIg(Publisher.java:806)
at org.hl7.fhir.igtools.publisher.Publisher.execute(Publisher.java:667)
at org.hl7.fhir.igtools.publisher.Publisher.main(Publisher.java:6932)
Jens Villadsen (Mar 25 2020 at 14:11):
nop ... it does not build locally
Jens Villadsen (Mar 25 2020 at 14:11):
thats the point
Josh Mandel (Mar 25 2020 at 14:11):
Oh -- OK. I think I misunderstood your question ;-)
Josh Mandel (Mar 25 2020 at 14:11):
I was just trying to make sure your branches were auto-published (which they are -- as long as they don't have / in their name).
Jens Villadsen (Mar 25 2020 at 14:11):
yep
Josh Mandel (Mar 25 2020 at 14:12):
Will need Grahame to comment on why the build itself is failing.
Jens Villadsen (Mar 25 2020 at 14:12):
that was also an issue
Jens Villadsen (Mar 25 2020 at 14:12):
so one down, one to go ;)
Jens Villadsen (Mar 25 2020 at 14:12):
@Grahame Grieve - your turn ;)
Jens Villadsen (Mar 25 2020 at 14:13):
does https://fhir.github.io/auto-ig-builder/builds.html only show master branches?
Jens Villadsen (Mar 25 2020 at 14:14):
and the version column might give me a hint that something is wrong as the version is 'null'
Josh Mandel (Mar 25 2020 at 14:14):
Yes, the summary is master branches only.
Josh Mandel (Mar 25 2020 at 14:15):
Your master branch doens't currently have content, so I wouldn't try to read too much into how that summary is populated.
Jens Villadsen (Mar 25 2020 at 14:15):
why not a line for each branch for each project?
Josh Mandel (Mar 25 2020 at 14:15):
Would be a welcome improvement -- originally the whole toolchain only supported master branches.
Jens Villadsen (Mar 25 2020 at 14:15):
just as jenkins ;)
Josh Mandel (Mar 25 2020 at 14:15):
I've set up some of the infrastructure so now the summary view should have access to all the data it needs to show details for all branches.
Josh Mandel (Mar 25 2020 at 14:16):
But this hasn't been a priority for me yet; would love help putting this together if you're game!
Josh Mandel (Mar 25 2020 at 14:17):
It'd be easy to start from scratch on the summary view using any JS framework of your choice. It just fetches static content from the build server and summarizes / displays it in tabluar form.
Jens Villadsen (Mar 25 2020 at 14:19):
sure - Im just a bit busy atm.
Jens Villadsen (Mar 25 2020 at 14:23):
with 3 kids at home (thx COVID19) and me and my wife being full time employees does not leave much time to spare
Josh Mandel (Mar 25 2020 at 15:03):
Yeah, it's a pretty tough time for "nice to have" feature development. Hopefully the lack of branch-builds on the dashboard won't stop you from using branches; you'll still get notifications in #committers/notification when branch builds finish, even if they aren't included on the dashboard. You can set up an "alert word" in Zulip for the name of your project to make sure you don't miss these.
Josh Mandel (Mar 25 2020 at 15:03):
See https://chat.fhir.org/#settings/alert-words
Grahame Grieve (Mar 25 2020 at 19:57):
@Jens Villadsen I'm not sure, reading this thread, whether there's still a problem here or not, and how I would reproduce it
Jens Villadsen (Mar 25 2020 at 19:58):
There is still a problem
Jens Villadsen (Mar 25 2020 at 20:00):
https://build.fhir.org/ig/hl7dk/dk-medcom-core-r4/branches/version_0.8/failure/build.log
Jens Villadsen (Mar 25 2020 at 20:01):
Check out the 0.8 branch
Jens Villadsen (Mar 25 2020 at 20:01):
And run the publisher
Jens Villadsen (Mar 25 2020 at 20:02):
That reproduces the bug
Jens Villadsen (Mar 25 2020 at 20:08):
@Grahame Grieve
Jens Villadsen (Mar 25 2020 at 20:56):
the log doesn't say anything about what file has caused the issue
Jens Villadsen (Mar 25 2020 at 21:52):
@Grahame Grieve the bug is here: https://github.com/hapifhir/org.hl7.fhir.core/blob/200f14922221520f13c123397cd4cf4b1b18fc6c/org.hl7.fhir.r5/src/main/java/org/hl7/fhir/r5/conformance/ProfileUtilities.java#L4591 - one of the index accesses are out of bounds
Grahame Grieve (Mar 25 2020 at 21:54):
one of your types has no code?
Jens Villadsen (Mar 25 2020 at 21:55):
cant say - if thats what the code says?
Jens Villadsen (Mar 25 2020 at 21:56):
in this particular project I'm not the author of the profiles ... I just help setting up the pipelines and aid in some general guidelines
Jens Villadsen (Mar 25 2020 at 21:57):
If I'd known which file that caused the issue, I would have corrected it myself
Grahame Grieve (Mar 25 2020 at 21:57):
I'll work on it tomorrow
Jens Villadsen (Mar 25 2020 at 21:57):
but from the log, I sort of have no clue
Jens Villadsen (Mar 26 2020 at 09:23):
@Josh Mandel how does the tooling handle that I've changed the URLS of the repo? Because I've just done that for two IG's.
Jens Villadsen (Mar 26 2020 at 10:32):
and now I don't see any updates on the IG builds anymore
Jens Villadsen (Mar 26 2020 at 12:11):
@Josh Mandel nevermind ... I found that removing the webhook and setting it up again solved the problem
Josh Mandel (Mar 26 2020 at 13:44):
Glad you figured this out! (Yeah, the initial webhook setup has you embed repo details. This could be improved, but as far as I know this is the first time it has come up.)
Jens Villadsen (Mar 26 2020 at 20:28):
@Grahame Grieve - any news on getting some more debug info on the bug?
Jens Villadsen (Mar 26 2020 at 20:46):
(the current log is located at https://build.fhir.org/ig/hl7dk/dk-medcom/branches/version_0.8/failure/build.log)
Grahame Grieve (Mar 26 2020 at 20:51):
Looking at this is scheduled sometime today if I can find time (supporting a go live on a covid-19 related integration)
Jens Villadsen (Mar 26 2020 at 22:39):
@Grahame Grieve https://github.com/HL7/fhir-ig-publisher/pull/66
Jens Villadsen (Mar 26 2020 at 22:40):
ProfileUtilities has issues on sorting differential it appears
Jens Villadsen (Mar 27 2020 at 19:58):
@Grahame Grieve IG is now merged to master with a bit more logging. I've checked out org.hl7.fhir.core and reproduced the bug that my PR sort of circumvents in the publisher. Build log can be found at http://build.fhir.org/ig/hl7dk/dk-medcom/failure/build.log
Jens Villadsen (Mar 28 2020 at 09:22):
And the problem is in line https://github.com/hapifhir/org.hl7.fhir.core/blob/200f14922221520f13c123397cd4cf4b1b18fc6c/org.hl7.fhir.r5/src/main/java/org/hl7/fhir/r5/conformance/ProfileUtilities.java#L4588 - and not as previous stated in L4591
Grahame Grieve (Mar 28 2020 at 11:05):
I didn't get any error there? how to reproduce?
Jens Villadsen (Mar 28 2020 at 11:13):
http://build.fhir.org/ig/hl7dk/dk-medcom/failure/build.log
Jens Villadsen (Mar 28 2020 at 11:13):
git clone https://github.com/hl7dk/dk-medcom
Jens Villadsen (Mar 28 2020 at 11:14):
The PR is here: https://github.com/HL7/fhir-ig-publisher/pull/66
Jens Villadsen (Mar 28 2020 at 11:16):
Before it crashes, I can see through debugging that the error list contains the following message: StructureDefinition profile http://medcom.dk/fhir/StructureDefinition/medcom-hospital-notification-message: Differential contains path Bundle.entry which is not found in the base
Jens Villadsen (Mar 28 2020 at 11:17):
And that is probably correct - so may I suggest that the list of detected errors are dumped to the console before exiting?
Jens Villadsen (Mar 28 2020 at 11:21):
regarding the line number - I debugged it on the org.hl7.fhir.core on commit 5097ab9a8b1ec62e49f01e17e37a41ea536366ba which changed the line numbers a bit. The error is indeed on the statement ElementDefinition ed = cmp.snapshot.get(child.getBaseIndex());as cmp.snapshot.get is empty
Jens Villadsen (Mar 28 2020 at 11:21):
@Grahame Grieve let me know if you have issues reproducing it.
Jens Villadsen (Mar 29 2020 at 06:55):
@Grahame Grieve ?
Grahame Grieve (Mar 29 2020 at 09:24):
sorry, I was having a life
Grahame Grieve (Mar 29 2020 at 19:20):
so it builds fine for me. Is there a branch that fails?
Jens Villadsen (Mar 29 2020 at 19:52):
builds fine now - ...
Jens Villadsen (Mar 29 2020 at 19:53):
but there was an error in the profiles that caused the publisher to not behave all great
Grahame Grieve (Mar 29 2020 at 20:22):
looking at the PR... I'm not going to approve something that simply suppresses errors
Jens Villadsen (Mar 30 2020 at 19:47):
you prefer crash and burn in this case?
Grahame Grieve (Mar 30 2020 at 19:56):
it doesn't crash and burn right now, right? I tink I'd rather crash and burn, yes, than hide problems
Jens Villadsen (Mar 30 2020 at 19:57):
so currently it actually hides the problem ...
Jens Villadsen (Mar 30 2020 at 19:57):
I'll change the PR in a moment
Jens Villadsen (Mar 30 2020 at 19:58):
as the error list is swallowed
Jens Villadsen (Mar 30 2020 at 20:17):
check the PR now
Jens Villadsen (Mar 30 2020 at 20:21):
and run it against https://github.com/hl7dk/dk-medcom with SHA f2360b6731648b0b1d283040bd028e444296a013
Jens Villadsen (Mar 30 2020 at 20:30):
that proves the bug - and shows that the PR improves the error message
Last updated: Apr 12 2022 at 19:14 UTC
