Stream: questionnaire
Topic: SDC IG build down?
Paul Lynch (Jan 08 2020 at 18:50):
http://build.fhir.org/ig/HL7/sdc gives me an odd directory listing, and other links are 404s.
Lloyd McKenzie (Jan 08 2020 at 19:07):
The CI build is unhappy right now. Hopefully fixed tomorrow once Grahame is back in action
Paul Lynch (Jan 08 2020 at 19:09):
Ideally, if the build breaks, it would not deploy to the public version of the site that people are using.
Lloyd McKenzie (Jan 08 2020 at 19:10):
@Josh Mandel Is that something that's possible to do? Retain the old copy if the CI build fails?
Josh Mandel (Jan 08 2020 at 20:24):
Sure -- though I'd definitely appreciate help making that change (it's not trivial, because the service receiving build content doesn't currently understand the content, and the service generating build content doesn't have access to previous build results).
Josh Mandel (Jan 08 2020 at 20:24):
In general the preferred workflow would be to make changes in branches and merge them when they're not broken.
Josh Mandel (Jan 08 2020 at 20:24):
In other words, keeping master healthy prevents broken CI content.
Grahame Grieve (Jan 14 2020 at 02:32):
that's not so useful when the problem is that the infrastructure collapges
Josh Mandel (Jan 14 2020 at 04:13):
Hmm, like if the infrastructure breaks between testing changes in a branch and merging them into master?
Josh Mandel (Jan 14 2020 at 04:14):
Anyway, maybe we can have a canonical place where the filed build logs go, like a subdirectory of the successful build output?
Grahame Grieve (Jan 14 2020 at 04:17):
which logs?
Josh Mandel (Jan 14 2020 at 14:15):
Build error log, and any of other output that you'd want to post in the case of a failure
Grahame Grieve (Jan 14 2020 at 19:06):
so if the build fails completely, put some kind of error log in place instead?
Grahame Grieve (Jan 14 2020 at 19:06):
so if the build fails completely, put some kind of error log in place instead?
Grahame Grieve (Jan 14 2020 at 19:07):
so if the build fails completely, put some kind of error log in place instead?
Paul Lynch (Jan 14 2020 at 19:09):
Why not reverse the merge commit on master if the build fails, to get back to the previous working version?
Josh Mandel (Jan 14 2020 at 19:11):
Do you mean automatically? This could cause some troubling / unstable behavior, and it's probably not desirable if it's a build infrastructure problem (rather than a content problem) that caused the failure -- because in that case, rolling back the commit wouldn't actually fix the build.
Josh Mandel (Jan 14 2020 at 19:14):
If the build fails (completely or otherwise), the question is: what's the desired behavior. Right now we upload the complete (failed) build context, including output, logs, and source materials -- and replace any published content with this. I was thinking a better step might be to take this whole package and upload it to a subdirectory of any previous successful build (if there is any). Basically this would mean:
- At any time at most one successful build output would be published (for a given branch)
- Any any time, at most one failed build output would be published
- At any given time, there could be a (most recent) success output and a (most recent) failure output, both published
Josh Mandel (Jan 14 2020 at 19:14):
If this sounds like the right outcome, we can work towards that. It's not entirely trivial, but should be doable. I'd love help if anybody is interested in rolling up their sleeves on the build infrastructure.
Grahame Grieve (Jan 14 2020 at 19:18):
what should happen if there's never been a successful build? that's an important case because it's the onramp step
Grahame Grieve (Jan 14 2020 at 19:20):
The other thing to say is that pretty much all I could aim to do in case of a total failure is to get the console log output dumped, but the host that runs the IG Publisher is in a better place to do that already
Paul Lynch (Jan 14 2020 at 19:29):
Why upload anything if the build fails? Is there no place to see the errors other than the public website? (As you can tell, I am not familiar with the build infrastructure.) I would think the process would look like:
1) Generate the new version of the web pages on a staging server
2) Confirm that the new version is good (no build failure; selected pages look okay)
3) Copy the pages from staging to production
4) Move the old version on production to a ".bak" directory
5) Move the new version into place
But, I am sure there other considerations of which I am not aware.
Grahame Grieve (Jan 14 2020 at 19:32):
this is a most useful page: https://fhir.github.io/auto-ig-builder/builds.html
Josh Mandel (Jan 14 2020 at 19:43):
If there has never been a successful build, then there's zero successful build hosted, and zero or one failure.
Josh Mandel (Jan 14 2020 at 19:45):
I'm thinking that in terms of file layout on our web server we'd have something like
example-ig/ ├── failure │ ├── errors.log │ ├── ig.json │ └── output │ └── other-source-files ├── index.html ├── success-file-1 └── success-file-2
Josh Mandel (Jan 14 2020 at 19:45):
So if you just navigate to the URL for the IG, you'd see the successful build; if you navigate to /failure, you'd get failure details (if any).
Josh Mandel (Jan 14 2020 at 19:46):
The "only publish good builds" approach you described in step (2) above @Paul Lynch doesn't explain how failure logs get published.
Paul Lynch (Jan 14 2020 at 19:49):
I was thinking that the failure logs would be viewable on the "staging server". But, I was also thinking that everything was published at once, which I see by Grahame's link that that is not the case, so things are more complex than I was thinking.
Last updated: Apr 12 2022 at 19:14 UTC
