FHIR Chat · FHIR Build choked up · committers

Stream: committers

Topic: FHIR Build choked up

Grahame Grieve (Dec 17 2021 at 02:32):

Hey @Josh Mandel I expect that the answer is no, but the Main FHIR build is very choked up. I've been processing PRs - which no one has been doing. I have 20 something builds going on, and it's choked up the system - the builds are all waiting for a free agent. They've all passed, and I'm going to merge the PRs - another 20 builds, none of which I care about, and it'll probably take most of the day. And I have a long way to go yet, another 25 PRs to process

is there any easy way to turn off builds on merge for a while?

Grahame Grieve (Dec 17 2021 at 02:49):

no, it's way worse than I realised - each successful merge forces a rebuild on any other PR. I can't do anything in parallel - I'm going to be processing PRs for days.

Grahame Grieve (Dec 17 2021 at 02:49):

given that, I don't see how i can get a R5 snapshot out.

Grahame Grieve (Dec 17 2021 at 02:49):

I suppose I can just ignore them and invalidate them all (I'm trying to merge them so I can do a sweeping change to the source)

Lloyd McKenzie (Dec 17 2021 at 04:07):

I think should be able to just temporarily remove the 'fhir' project from Azure pipelines here: https://github.com/organizations/HL7/settings/installations/9677264

Lloyd McKenzie (Dec 17 2021 at 04:08):

That means there won't be any use of Azure at all for a bit

Lloyd McKenzie (Dec 17 2021 at 04:09):

You might want to temporarily remove commit access from the FHIR commiters group when you do that.

Lloyd McKenzie (Dec 17 2021 at 04:09):

Then go crazy and merge, make the build work, then turn Azure back on again and grant permissions again.

Lloyd McKenzie (Dec 17 2021 at 04:09):

I am, however, someone who doesn't really understand how all this stuff works, so my proposal might be horrifyingly ill-advised :)

Josh Mandel (Dec 17 2021 at 04:14):

You @mentioned me here but I'm not currently managing the build pipeline. Is that @Mark Iantorno ?

Josh Mandel (Dec 17 2021 at 04:16):

That said, I'm trying to follow along here .

I'm not sure what "builds on merge" is -- where is that configured, or who configured it?

Lloyd McKenzie (Dec 17 2021 at 04:37):

I presume it's something similar to what you set up for me w/ JiraSpecArtifacts. When there's a merge, previous pull requests must be updated (which generally just requires pushing a button) and then they re-build to see if they're valid to merge. It's just that w/ JiraSpecArtifacts, that process takes about 30 seconds. With FHIR core, it's closer to 40 minutes...

Grahame Grieve (Dec 17 2021 at 05:06):

you set it up in the first place, @Josh Mandel

Josh Mandel (Dec 17 2021 at 14:39):

you set it up in the first place

I did, but it has since been migrated entirely from that original setup. In particular the "build on merge" is not ringing any bells, and I don't think I have access to the Pipelines accounts to review settings.

Josh Mandel (Dec 17 2021 at 14:40):

there's a merge, previous pull requests must be updated (which generally just requires pushing a button)

That's different! That's just a rule saying branches need to be current in github before they can be merged. Nothing automatically triggers every PR to build with every commit.

Josh Mandel (Dec 17 2021 at 14:41):

From what Grahame was describing above, it sounds like a cascade of O(n^2) automated builds were being triggered on the path to completion (all unmerged PRs building every time any one PR is merged, meaning N + (N-1) + (N-2) + ...). when a controlled merging process would let you Merge PR1 into main, Merge main into PR2, Merge PR2 into main, Merge main into PR3, Merge PR3 to main... which is O(n) work.

Let me know if I misunderstood.

Josh Mandel (Dec 17 2021 at 14:47):

If nothing automated is happening and you just want to merge stuff, the most controlled process would be to locally do...

• make a merging-cxn-2022-21 branch
• merge all desired PR branches into it (using merge commits, i.e. git merge Composition-Change-11-18-2021 and so on...
• test and build locally
• push to GH, and merge this big honking branch to get all the changes into master

At that point, I think all the open PRs will notice they've been merged and disappear (otherwise we can just close 'em).

Josh Mandel (Dec 17 2021 at 14:48):

That said, just to be complete: on the github configuration side, the reason this appears:

image.png

is that https://github.com/HL7/fhir/settings/branch_protection_rules/2277268 is configured like:

image.png }

Of course this could be turned off, but the approach I described above sounds more likely to avoid broken builds and confusion.

Vassil Peytchev (Dec 17 2021 at 15:27):

@Grahame Grieve let me know if you need someone to consolidate the PRs into a single merge - I can probably do it by 3 pm US Eastern time.

Grahame Grieve (Dec 17 2021 at 18:26):

thanks but I'll try myself

Grahame Grieve (Dec 17 2021 at 18:32):

so say I'd like to merge this PR locally: https://github.com/HL7/fhir/pull/1080

Grahame Grieve (Dec 17 2021 at 18:33):

I think this looks like a good recipe: https://stackoverflow.com/questions/21353656/merge-git-repo-into-branch-of-another-repo/21353836

Grahame Grieve (Dec 17 2021 at 18:34):

so I do git

checkout -b gg-202112-multi-merge
git remote add ward git@github.com:wardweistra/fhir.git
git remote update

And I get:

Fetching origin
Fetching ward
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
error: Could not fetch ward

Did I do something wrong?

Vassil Peytchev (Dec 17 2021 at 19:01):

Are you usually working with SSH-based or https-based remotes? The above worked for me. Did you try git remote add ward https://github.com/wardweistra/fhir.git?

Grahame Grieve (Dec 17 2021 at 19:01):

Are you usually working with SSH-based or https-based remotes? The above worked for me.

I have no idea what that means

Grahame Grieve (Dec 17 2021 at 19:01):

Did you try git remote add ward https://github.com/wardweistra/fhir.git

yes that does work

Josh Mandel (Dec 17 2021 at 19:02):

Excellent.

Vassil Peytchev (Dec 17 2021 at 19:09):

Grahame Grieve said:

Are you usually working with SSH-based or https-based remotes? The above worked for me.

I have no idea what that means

It is the difference between this:
github_clone_ssh.png
and this:
github_clone_https.png

WIth SSH, you upload your SSH public key to github, and won't have to worry about passwords.

Grahame Grieve (Dec 17 2021 at 19:13):

well, I haven't done the public key thing, so I guess I don't want to do that

Josh Mandel (Dec 17 2021 at 19:24):

Cool, so if you do git remote -v you should see origin URLs for your remotes with https://.

Chris Moesel (Dec 17 2021 at 19:45):

BTW -- if you just want to merge a PR locally, GitHub actually provides instructions on how to do this within each PR (as long as it is in a mergeable state). To the far right of the merge button, you should see a tiny link that says "command line instructions". Click that and it expands a view with instructions on how to do it locally.

Here's a screenshot of the instructions I see for merging a SUSHI PR from our friend Jose:
image.png

NOTE: The first bit of the instructions is for testing locally on a different branch. The last bit of the instructions is for merging it to your local master branch and pushing it to the remote (which you should only do when you're ready). It will close the PR.

Grahame Grieve (Dec 17 2021 at 19:48):

right. I usually do that. but this triggers the github actions, which is what I'm trying to avoid in this case

Josh Mandel (Dec 17 2021 at 19:49):

Just skip the "push" step, and repeat for each PR you want... and then push

Grahame Grieve (Dec 17 2021 at 19:49):

though also, command line instructions usually end up leaving whoever follows them in an unrecoverable mess

Chris Moesel (Dec 17 2021 at 19:51):

Ha. Yeah, if you merge something into your local master and then decide you need to back it out, sometimes that can be a little messy if you don't do it exactly right.

Grahame Grieve (Dec 17 2021 at 20:26):

https://github.com/HL7/fhir/pull/1555

Grahame Grieve (Dec 17 2021 at 20:26):

now I just have to get it to build.

Josh Mandel (Dec 17 2021 at 20:34):

It passes locally??

Grahame Grieve (Dec 17 2021 at 20:42):

oh yes I just need to get a new set of code out so it can build on github, and that requires sonatype to work - we're having pipeline problems

Grahame Grieve (Dec 20 2021 at 11:01):

all sorted - thanks for the help.

Grahame Grieve (Dec 20 2021 at 11:01):

and published too

Last updated: Apr 12 2022 at 19:14 UTC