FHIR Chat · Hello World · committers/git-help

Stream: committers/git-help

Topic: Hello World

Josh Mandel (Sep 04 2018 at 17:57):

Hi @Angelo Kastroulis @Andy Stechishin @Steve Munini @Vadim Peretokin @Geoff Low @Nagesh Bashyam @Robert Horn @Abigail Watson @Joel Schneider -- I'm subscribing you here because you mentioned you might be willing to help out in answering community questions about how to use git and GitHub. We're finally in the process of transitioning the FHIR core spec development from svn to GitHub, and I'm setting up this channel for support. (We'll be doing a proper, broader announcement later this week, but I wanted to lay the groundwork first.)

The quick ask of you, if you're still willing, is to set up notifications for this #committers/git-help channel and lend a hand when people come with questions.

(Cc @Grahame Grieve @Lloyd McKenzie @Ewout Kramer )

Steve Munini (Sep 04 2018 at 20:17):

Happy to help Josh!

Abbie Watson (Sep 04 2018 at 20:32):

Sure. If we've got FHIR.js and core FHIR spec all on GitHub and messaging connected via git hooks to Zulip streams, maybe we should set up Meteor on FHIR as well.....

Nagesh Bashyam (Sep 05 2018 at 03:40):

:thumbs_up: :thumbs_up: Happy to help

Vadim Peretokin (Sep 05 2018 at 10:58):

Awesome. What is the status of the transition atm?

edit: https://github.com/hl7-fhir/fhir-svn/wiki/Get-Started-with-FHIR-on-GitHub

Josh Mandel (Sep 05 2018 at 12:37):

That's the right reference. We're planning to cut over at the end of this week, and trying to get things set up before then

Geoff Low (Sep 05 2018 at 13:23):

Also happy to assist. Couple of questions: are we using the protected branches feature, with minimum N reviewers and status checks (could we setup travis-ci to automatically review changes and make sure they don't break the build)?

Grahame Grieve (Sep 05 2018 at 13:24):

we'll be doing that - normal commit is to a branch that the build checks and then automatically commits to trunk

Geoff Low (Sep 05 2018 at 13:34):

Ah, sorry - didn't read far enough down the supplied document!

Josh Mandel (Sep 05 2018 at 13:37):

To clarify, there's no "auto commit to the truck" -- current process we're standing up is the PR can't be merged until status checks complete, and then it can be merged.

Josh Mandel (Sep 05 2018 at 13:37):

But the merge to master requires pressing the button.

Grahame Grieve (Sep 05 2018 at 13:40):

really? that sounds painful.

Grahame Grieve (Sep 05 2018 at 13:40):

some days I do 100s of commits

Josh Mandel (Sep 05 2018 at 13:44):

In general these would be grouped into relevant feature sets.

Grahame Grieve (Sep 05 2018 at 13:45):

I don't know what that means

Josh Mandel (Sep 05 2018 at 13:46):

oh, like normally you would create a feature Branch to work on a feature. You might do hundred commits in the course of developing that feature, and then you would merge the branch once the feature was complete.

Grahame Grieve (Sep 05 2018 at 13:47):

that's kind of not the workflow. I'm typically processing typos and stuff like that.

Josh Mandel (Sep 05 2018 at 13:47):

I'm not saying it's impossible to have a workflow with auto merging, but there is no built-in support for that kind of workflow, so we would have to create some kind of listener component and call the merge API when certain conditions were met -- and we would need a way to tell which PRs were intended to the auto merge. I've never seen this in the wild.

Grahame Grieve (Sep 05 2018 at 13:47):

i suppose I could do it that way

Josh Mandel (Sep 05 2018 at 13:49):

Yeah -- if you read through the write-up Wiki document linked above, and this looks unworkable, we should step back and think about how to change it.

Josh Mandel (Sep 05 2018 at 13:51):

Stuff like https://mergify.io looks quite flexible although I'm reluctant to take a dependency on it.

Grahame Grieve (Sep 05 2018 at 13:52):

I'm not sure. most of the time, the PR workflow will be fine. It's not many days that I go through 100s of tasks all over the place

Josh Mandel (Sep 05 2018 at 13:53):

Ok. If you are willing to try it this way, I would say we should give it a shot. We can always adjust the process as we learn more -- we can make it as complicated as we need to :-)

Grahame Grieve (Sep 05 2018 at 13:54):

yes we give it a shot

Geoff Low (Sep 05 2018 at 14:01):

yeah, we tend to bundle changes into branches, which are usually linked to jira tickets; so you bundle the effort for a particular fix or feature into a branch; then you push that bundle of work into GH and raise a PR against the master or develop branch from the feature branch.

Grahame Grieve (Sep 05 2018 at 14:04):

and lots of tasks I work on, that's how it works. But I also process 100s of tasks that are just tiny little text or grammar fixes. Can presently average 2:30 secs for these

Josh Mandel (Sep 05 2018 at 14:08):

what I might recommend for those cases is to do a commit for each one (if you want the granularity) and then reference the GForge issue in the commit message(s). But push them all up as a single branch so you can group together the tests+merge into a single step that covers all 100 of them.

Josh Mandel (Sep 05 2018 at 14:09):

In other words to pull request would basically be for "a bunch of typos I fixed this afternoon".

Geoff Low (Sep 05 2018 at 14:12):

yup, that's it

Grahame Grieve (Sep 07 2018 at 21:42):

jawalonoski deleted the branch jwalonoski_clean_old_svn_files.

umm, we have to go back and delete branches?

Bryn Rhodes (Sep 07 2018 at 21:45):

When you merge a PR, you get the option to delete the branch. I generally do because it keeps the branches down.

Josh Mandel (Sep 07 2018 at 21:47):

We can introduce automatic cleanup at some point.. but the branches also don't harm anything so it's more of a nice to have.

Grahame Grieve (Sep 07 2018 at 22:43):

so I just followed the instructions here: https://github.com/HL7/fhir/wiki/Get-Started-with-FHIR-on-GitHub. They are perfectly horrible and are 10x more work than working under svn. I will hope that we can make better instructions somewhere.

Grahame Grieve (Sep 07 2018 at 22:43):

the most horrible bit is the named branches. is there some expectation that we'll have differently named branches for each piece of work?

Grahame Grieve (Sep 07 2018 at 22:56):

so my PR just failed. I did a build, it worked. Then I committed using git, and it truncated a file

Grahame Grieve (Sep 07 2018 at 22:56):

it appears to be truncated at the point where I quitted from git diff.

Grahame Grieve (Sep 07 2018 at 22:56):

git did not notify that it was trashing my file...

Grahame Grieve (Sep 07 2018 at 23:10):

so I launched a pull request through tortoisegit. I got this as outcome

Grahame Grieve (Sep 07 2018 at 23:10):

The following changes since commit b0679f740d350ffe085eb4d65f64ad629a99eb96:

  compile fix, and improve git ignore (2018-09-08 08:24:34 +1000)

are available in the Git repository at:

  https://github.com/HL7/fhir gg-work

for you to fetch changes up to c76f807961e81fcfbe7d69a856fa70769e6f68ab:

  fix loinc.xml (2018-09-08 09:07:30 +1000)

----------------------------------------------------------------
Grahame Grieve (2):
      fix up git ignored
      fix loinc.xml

 .gitignore | 1 +
 1 file changed, 1 insertion(+)

Josh Mandel (Sep 07 2018 at 23:10):

Can you clarify whether the horribleness issue is the instructions or the actual procedures themselves?

Grahame Grieve (Sep 07 2018 at 23:11):

I don't understand the "for you to fetch changes up" line - is this something I need to do? or something I could do if I wanted?

Grahame Grieve (Sep 07 2018 at 23:11):

the procedures are horrible.

Grahame Grieve (Sep 07 2018 at 23:14):

and now I'm lost again

Josh Mandel (Sep 07 2018 at 23:14):

I'm not sure exactly what that message is from, but let's look at GitHub.

Grahame Grieve (Sep 07 2018 at 23:14):

I made a commit following the instructions, to gg-work

Grahame Grieve (Sep 07 2018 at 23:14):

then I created a pull request

Josh Mandel (Sep 07 2018 at 23:14):

https://github.com/HL7/fhir/pull/18

Grahame Grieve (Sep 07 2018 at 23:14):

that failed, because git truncated my file

Grahame Grieve (Sep 07 2018 at 23:14):

now I made a new commit to the branch that fixed the file git trashed

Josh Mandel (Sep 07 2018 at 23:15):

Can you clarify what you mean by truncated a file?

Josh Mandel (Sep 07 2018 at 23:15):

I see 3 commits currently on not pull request Branch. What additional changes are you expecting?

Grahame Grieve (Sep 07 2018 at 23:15):

now i want to do a PR on the to move changes from gg-work to the head, and I can't figure out how to do it

Grahame Grieve (Sep 07 2018 at 23:16):

git cut the loinc.xml file off at bout line 200

Josh Mandel (Sep 07 2018 at 23:16):

So you have to pull request already created. Now it's just a question of merging it.

Grahame Grieve (Sep 07 2018 at 23:16):

I don't have it already created.

Josh Mandel (Sep 07 2018 at 23:16):

I don't think git cut the file off. I think you're just looking at a preview. Where are you seeing this?

Grahame Grieve (Sep 07 2018 at 23:16):

it completely did get cut off

Josh Mandel (Sep 07 2018 at 23:17):

https://github.com/HL7/fhir/pull/18 has a merge button that will become active when the build status checks complete.

Grahame Grieve (Sep 07 2018 at 23:17):

umm

Grahame Grieve (Sep 07 2018 at 23:18):

so I didn't understand. The pull request was magically brought back to life after it failed.. because ... why?

Josh Mandel (Sep 07 2018 at 23:19):

the PR is requesting that a particular Branch be merged. If you keep adding commits to the branch, each time you add a new commit the status check will run again.

Josh Mandel (Sep 07 2018 at 23:19):

At some point when you are happy with a set of commits currently on the branch you can merge them into master.

Josh Mandel (Sep 07 2018 at 23:20):

Or at any point you can give up and close the PR and optionally delete the branch.

Grahame Grieve (Sep 07 2018 at 23:21):

so if I merge into the master, and then make new commits to the branch, then what?

Josh Mandel (Sep 07 2018 at 23:21):

Well that would not be a recommended practice. GitHub will consider the pull request to be merged so you would have to open a new one. you could technically do this (opening a new requests on the same persistent branch) but it doesn't have much to recommend it.

Josh Mandel (Sep 07 2018 at 23:22):

Better to keep them short lived

Grahame Grieve (Sep 07 2018 at 23:22):

I guess this answers my earlier question:

the most horrible bit is the named branches. is there some expectation that we'll have differently named branches for each piece of work?

Grahame Grieve (Sep 07 2018 at 23:23):

that means the answer is yes, and I have a real problem with it. Perhaps it's ok for you, who rarely commit to the build, but I can make 100s of commits a day

Josh Mandel (Sep 07 2018 at 23:23):

You could have special dispensation to commit directly to master if you want :-)

Josh Mandel (Sep 07 2018 at 23:24):

But this kind of workflow is very typical across most of the software development practices that I've seen.

Grahame Grieve (Sep 07 2018 at 23:24):

I think I want that for special cases, but not normally

Josh Mandel (Sep 07 2018 at 23:24):

(I would love to hear from others here.)

Josh Mandel (Sep 07 2018 at 23:26):

I do want to emphasize that a typical workflow might involve hundreds of commits on a single feature branch. And you merge that all as the completed feature, and then you're done. I know this doesn't describe everything you do, but it is a good mindset to have for the things where it does fit. With that, I am off to dinner with Carrie :-)

Grahame Grieve (Sep 07 2018 at 23:27):

it just feels to me like the manual trickiness here - going from a simple 'commit, write change notes, done' to 'name branch, switch to it, commit, push named branch by name, make a pr, watch for the pr to complete, merge' turns a 15 second operation into something that takes much longer - 3-4 minutes cumlatively? (and I can't see how I can interlace work either, which I do all the time)

Grahame Grieve (Sep 07 2018 at 23:28):

hundreds of commits on a single feature branch

Grahame Grieve (Sep 07 2018 at 23:28):

is completely not my normal workflow. sometimes I do something that could look like that.

Grahame Grieve (Sep 07 2018 at 23:28):

but it's rare.

Grahame Grieve (Sep 08 2018 at 00:35):

something else I don't know what I did wrong:

Grahame Grieve (Sep 08 2018 at 00:35):

I made a commit to a new branch:
grahamegrieve pushed 4 commits to branch gg-no-svn.
compile fix, and improve git ignore (b0679f7)
fix up git ignored (4c84b9e)
fix loinc.xml (c76f807)
remove references to svn throughout code base (cc40a26)

Grahame Grieve (Sep 08 2018 at 00:36):

that's all my commits from the other branch. I thought I had reset my work area, but obviously not...

Lloyd McKenzie (Sep 08 2018 at 00:48):

One annoying thing about many branches is that even if we clean up the main repository, the branches stick around on the local machine until they're manually cleaned up there.

Lloyd McKenzie (Sep 08 2018 at 00:49):

(or at least I haven't figured out how to make them go poof automatically)

Josh Mandel (Sep 08 2018 at 13:31):

https://stackoverflow.com/a/41611648 may be helpful if you want this behavior

Lloyd McKenzie (Sep 08 2018 at 15:50):

Ok. Is there any reason this wouldn't be something everyone would want? I.e. any reason to keep local versions of branches that have already been applied to the master? If not, can we add this to the Git setup instructions - including how to do it using non-commandline :)

Grahame Grieve (Sep 08 2018 at 20:52):

that stack overflow question feels like a powerful statement of what I think is wrong about git

Grahame Grieve (Sep 09 2018 at 11:07):

Current git hit list:
- how do you maintain a single local clone, but commit to multiple different branches
- how do we replace the build number system
- how do we tell people how to know when a commit makes it's way onto build.fhir.org
- how to auto-merge commits from authorised users
- how to auto-delete local branches that are dead

Rob Hausam (Sep 09 2018 at 14:42):

gave my answer for #1 on committers

Josh Mandel (Sep 09 2018 at 15:29):

For removing unused branches, there's a comment above -- would be good to know if this achieves the desired behavior.

Josh Mandel (Sep 09 2018 at 15:31):

For "auto merging", this needs a bit more explanation; how would a Committer indicate which pull requests are intended for automatic merging, vs work in progress? This is especially important as we add more review process.

Lloyd McKenzie (Sep 09 2018 at 15:31):

@Rob Hausam for #1, can you walk through the workflow for that? I have a local repository. It contains 3 changed files. I want to commit/push/merge one of them. The other 2 aren't ready to share yet and I don't want there to be any risk of those changes being pushed and merged

Lloyd McKenzie (Sep 09 2018 at 15:32):

@Josh Mandel The committer wouldn't indicate. Nothing gets pushed unless it's ready to merge. The Git server would have to determine based on what had been committed whether additional review was needed.

Josh Mandel (Sep 09 2018 at 15:32):

For telling people when changes are published, watching #committers/notification for a successful build status on master should help -- once announced, the build is published to build.fhir.org

Lloyd McKenzie (Sep 09 2018 at 15:34):

The "when changes are published" problem is how you say the equivalent of "watch for 17234 to successfully build, then grab the latest changes" where 17234 needs to be something human-recognizable - i.e. not a hash

Lloyd McKenzie (Sep 09 2018 at 15:36):

BTW - I presume the notion of pushing a work in progress to a branch is if you're doing shared development on a branch or are doing major work you don't want to lose if something bad happens locally? We could, perhaps, have a naming convention for such branches. In SVN, that would have been done with a true "branch" and in practice, I'm not sure we ever did that.

Josh Mandel (Sep 09 2018 at 16:31):

Why can't someone watch for a hash in the notification channel?

Josh Mandel (Sep 09 2018 at 16:33):

That is, Why would it have to be an integer?

Josh Mandel (Sep 09 2018 at 16:34):

Re: sharing work in progress, yes, the key use case would be review, whether via automated checks or individual human reviewers

Lloyd McKenzie (Sep 09 2018 at 16:56):

Two main reasons - hashes are gobbledygook that's hard for a human brain to match; a sequential number gives you a sense of how long you need to wait

Grahame Grieve (Sep 09 2018 at 20:40):

Why can't someone watch for a hash in the notification channel?

because they're not subscribed to Zulip?

Grahame Grieve (Sep 09 2018 at 20:41):

re: auto-merge- the discussion here seems to be about auto-pull request. That's not what I thought we were talking about:

how would a Committer indicate which pull requests are intended for automatic merging, vs work in progress

Why would you have a PR on work in progress?

Bryn Rhodes (Sep 09 2018 at 20:44):

Why not just define a build number, that's what we're really doing, we just used the SVN revision number as a convenient shorthand. Moreover, there's already build tooling that depends on that build number, seems like the easiest thing to do to support the transition is to continue that, but since the source control no longer supports it, we need to maintain it. So build a simple process that increments the build number in a master version file in the build, and commits the change. That's a release. Whether we automatically do that on every commit is another question.

Grahame Grieve (Sep 09 2018 at 20:44):

@Rob Hausam :

The only operation you need for switching to commit to different branches of your single local clone is 'git checkout', in combination with committing first before you checkout and switch to a different branch or using 'git stash' to save the state of your work (including 'git stash -u' or 'git stash -a' if you also want to save the state of untracked or even "ignored" files).

My workflow is: I want switch branches so I can commit, not commit so I can switch

Grahame Grieve (Sep 09 2018 at 20:45):

So build a simple process that increments the build number in a master version file in the build, and commits the change

That's what I wanted, but I was assured it was impossible.

Grahame Grieve (Sep 09 2018 at 20:46):

it certainly can't be manual though - must be maintained by the infrastructure. But I think, in the end, that it isn't viable

Bryn Rhodes (Sep 09 2018 at 20:47):

It's impossible to have the git source control do it automatically like SVN did, but it's not impossible.

Bryn Rhodes (Sep 09 2018 at 20:47):

Yes, by manual I just mean you have to indicate that you want to do it. We could auto-wire it to the commit if we wanted, but that would mean every commit would have two commits, one for the change and one to increment the build number.

Lloyd McKenzie (Sep 09 2018 at 20:48):

Could we suppress notification of the second commit?

Bryn Rhodes (Sep 09 2018 at 20:48):

Well, what I'm wondering is whether we need to release on _every_ commit.

Bryn Rhodes (Sep 09 2018 at 20:49):

Should there be an "I want to release now" decision?

Lloyd McKenzie (Sep 09 2018 at 20:49):

Note that that still doesn't help in terms of telling peole what to look for in the CI build because when you commit, nothing would tell you what the number was going to be...

Lloyd McKenzie (Sep 09 2018 at 20:49):

The CI build releases everytime it succeeds (and increments even when it doesn't succeed). I don't know why we'd diverge from that.

Grahame Grieve (Sep 09 2018 at 20:49):

Should there be an "I want to release now" decision?

no; the CI build is the current. why complicate matters?

Bryn Rhodes (Sep 09 2018 at 20:53):

That's fair, but are we okay with every commit having another commit that increments the build number?

Bryn Rhodes (Sep 09 2018 at 20:54):

It would effectively double the size of the commit log. And it would mean that you'd have to pull after every commit.

Lloyd McKenzie (Sep 09 2018 at 20:56):

You have to pull before you do a push regardless, no? You wouldn't pull after every commit - that could be pulling 50+ times a day sometimes.

Bryn Rhodes (Sep 09 2018 at 20:56):

True, just before you push.

Josh Mandel (Sep 09 2018 at 21:05):

I guess the question now is whether this would work or not work. Obviously we can imagine ways that everything could work better -- and we can invest in building explicit tooling when needed. We can create a bot to notify you when you were build is ready, for example. But first we have to understand the problems that matter. :-)

Grahame Grieve (Sep 09 2018 at 21:06):

for me, the problems that matter are my hit list above

Lloyd McKenzie (Sep 09 2018 at 21:07):

I'll qualify that with "the problems sofar discovered that matter..." :)

Josh Mandel (Sep 09 2018 at 21:14):

The idea of building tools to generate follow-on commits to for every commit sounds like it'll be surprising. Software projects I'm familiar with use an explicitly release version then it matters. I'm very wary of trying to rewrite git to make it work like svn. We don't need to switch if we don't want to.

Grahame Grieve (Sep 09 2018 at 21:16):

Software projects I'm familiar with use an explicitly release version then it matters

Grahame Grieve (Sep 09 2018 at 21:16):

that's a bizarre statement for me. Because I'm asking questions that seem like they would be inevitable for any project, and there doesn't seem to be any answers

Lloyd McKenzie (Sep 09 2018 at 21:18):

With a continuous integration build, why would every commit not "matter"?

Rob Hausam (Sep 09 2018 at 21:23):

@Grahame Grieve I'm pretty sure that you can arrange your workflow so that you can "switch branches so I can commit" - if that's still what you want to do. You can use stash if you want to avoid making additional interim commits (whether or not it's "discouraged") - maybe that will work for you.

Josh Mandel (Sep 09 2018 at 21:23):

Matter for explicitly human versioning. Often in a continuous deployment scenario, you'll build a binary for each commit, labeling it with a git hash. Deployment can be made and rolled back via hash.

Bryn Rhodes (Sep 09 2018 at 21:24):

That's the same thing, it's just a much less useful label (especially given we already have a history with a sequential build number, if we now "switch" to github hashes, build tooling will forever have a "before this date look for this pattern" special case).

Lloyd McKenzie (Sep 09 2018 at 21:27):

I don't understand "Deployment can be made and rolled back via hash"

Lloyd McKenzie (Sep 09 2018 at 21:28):

We don't have any "deployment". If we want to undo a commit, we grab the original version and commit that. Everything's always a roll-forward.

Bryn Rhodes (Sep 09 2018 at 21:29):

We don't have any "deployment".

We do though, every time we publish the build, that's a deployment, and it's tagged with a build number that downstream tooling relies on to make decisions.

Josh Mandel (Sep 09 2018 at 21:29):

I'm just describing common patterns I see, in response to "why wouldn't every commit matter"

Lloyd McKenzie (Sep 09 2018 at 21:29):

The only "official" releases we have are the ballots and official publications. And those are snapshots taken of the master after imposing tight constraints that lock others from pushing/merging anything for a period of time.

Lloyd McKenzie (Sep 09 2018 at 21:31):

I guess I don't understand the explaination. In a CI build, who would roll anything back via hash and why?

Bryn Rhodes (Sep 09 2018 at 21:31):

Well, in a continuous delivery pipeline, the goal is to automate as much as possible of the deployment process. If you subsequently find an issue that made it farther down the pipe than it should have, you can roll it back.

Bryn Rhodes (Sep 09 2018 at 21:33):

That's true, but if the only thing that ever increments that number is the master build process, and it's only ever made as a single-file commit, that should be an auto-resolvable issue.

Bryn Rhodes (Sep 09 2018 at 21:35):

The number has to be in the build somewhere anyway, it's in the footer of every page.

Lloyd McKenzie (Sep 09 2018 at 21:35):

If you roll back, you don't have an incrementing version.

Grahame Grieve (Sep 09 2018 at 21:39):

I don't really understand rollback. what does it mean, and how can you know it's safe?

Rob Hausam (Sep 09 2018 at 21:50):

@Lloyd McKenzie For:

for #1, can you walk through the workflow for that? I have a local repository. It contains 3 changed files. I want to commit/push/merge one of them. The other 2 aren't ready to share yet and I don't want there to be any risk of those changes being pushed and merged

If I'm understanding the scenario (and I think I do), in that case I would use git add to stage the one changed file that I want to commit, and then use git commit to commit that file only (and you can push immediately afterward, or defer that until later as you prefer). Then, if I wanted to checkout and work on a different branch I would use git stash (with -u if either or both of those files are untracked) to save the work on the other 2 files and then checkout the other branch. Then when I'm ready to come back to that branch to keep working on those two files (and any others) I would checkout that branch and do git stash pop (or apply) to keep working on them where I left off before.

Lloyd McKenzie (Sep 09 2018 at 21:53):

I have 3 changed files. None of them require git add because all 3 files are already in source control. I'm never going to commit without pushing and I'm never going to push without merging. If I commit, it's because "this work is done and ready to add to the main build". (We can explore use-cases for committing without pushing or pushing without merging later, but the default for everyone at the start will be commit =push = merge (if build successful).

Lloyd McKenzie (Sep 09 2018 at 21:54):

I don't "want" to work on a different branch. But, it sounds like every time I push, I need to be on a different branch.

Rob Hausam (Sep 09 2018 at 21:58):

No, I think there's some misunderstanding here. First, you do need 'git add' to say what you want to be committed in the next commit (that doesn't happen without you saying so, regardless of whether the files are under version control). I agree that then the typical (or universal) workflow will be to push and merge what I've just committed. And you do not need to be on a different branch every time that you push. You can do as many commit - push command pairs as you want on a single branch (as long as there are further changes available to commit). I've done that lots of times.

Lloyd McKenzie (Sep 09 2018 at 21:58):

So what I think I'm hearing is the following:
1. make changes as you like in root.
2. When you're ready to commit some stuff:
a) stash your changes
b) create a new branch relevant to that set of changes
c) switch to that branch
d) unstash your changes to the current branch
e) commit the changes you want committed
f) re-stash the rest of the changes
g) pull from git to make sure you've got the latest, handling any merges necessary
h) push the commit and create a pull request
i) until we have auto-merge, go to the website and merge the pull request
j) checkout your original branch
k) unstash the uncommitted changes and keep working
l) trust that the branch you just committed will get deleted at some point in the future when the server cleans up its branch and that gets pushed down to your local store if you've got your config set up properly

Lloyd McKenzie (Sep 09 2018 at 21:58):

Is that correct?

Grahame Grieve (Sep 09 2018 at 21:58):

I would use git stash (with -u if either or both of those files are untracked) to save the work on the other 2 files and then checkout the other branch

So the work I'm doing is lost when I switch branches

Rob Hausam (Sep 09 2018 at 22:00):

the one issue is with generating pull requests - that does need to be from a different branch (not master)

Lloyd McKenzie (Sep 09 2018 at 22:00):

But that's addressed in my proposed workflow, no?

Rob Hausam (Sep 09 2018 at 22:00):

I think it is, Lloyd - yes (I just haven't read all of it yet)

Rob Hausam (Sep 09 2018 at 22:01):

@Grahame Grieve no, if you use git stash followed later by git stash pop none of the work will be lost

Lloyd McKenzie (Sep 09 2018 at 22:05):

When I do a pull in step g, that only effects the new branch, not my local master -correct? Or am I going to have worry about merging in step k too?

Rob Hausam (Sep 09 2018 at 22:10):

@Lloyd McKenzie I think you can simplify your proposed workflow substantially if you do the work that you're going to commit and push in a branch (not master) that you will then generate the pull request from (that should obviate the need for a-d, and some of the rest). I've seen recommendations and I agree that it's best not to make any changes directly in master (what you are calling root?), as it's easiest if that's left in a clean state so that any git pulls from upstream can always be fast-forward merged. From there you do do a git merge to any branch that you are doing work on where you want it to be updated to reflect the latest changes from others.

Lloyd McKenzie (Sep 09 2018 at 22:11):

Grahame's whole point is that he needs to do all his work in one branch because he needs to only build one branch.

Rob Hausam (Sep 09 2018 at 22:11):

he can do that if he wants to

Lloyd McKenzie (Sep 09 2018 at 22:11):

He can't sit and twiddle his thumbs waiting for 45 minutes while the build runs before he keeps working on other stuff

Lloyd McKenzie (Sep 09 2018 at 22:11):

how?

Lloyd McKenzie (Sep 09 2018 at 22:12):

We're talking local build here.

Grahame Grieve (Sep 09 2018 at 22:15):

I've never said this:

he needs to do all his work in one branch

what I said is "I need to do all my work in one local directory". I'm not following everything here completely, but I think those are different things

Lloyd McKenzie (Sep 09 2018 at 22:17):

Right. You need to work in one folder and kick off a build and keep working without having a bunch of files revert because you've switched branches and make the running build confused and either die or not reflect your changes.

Grahame Grieve (Sep 09 2018 at 22:18):

pretty much. I very frequently run the build basically continuously - stop and start again, or start as soon as it's done. I can have the build running >90% of the time

Lloyd McKenzie (Sep 09 2018 at 22:19):

The proposal I described above would still have to wait for the build to end - or at least get past its dependency on source files before you could do the stash/switch/unstash/.../switch back step

Lloyd McKenzie (Sep 09 2018 at 22:20):

But it would allow you to have multiple independent edits in your working folder and commit a subset of those in a branch without losing the rest.

Lloyd McKenzie (Sep 09 2018 at 22:20):

It is, however, a butt-ugly set of steps...

Lloyd McKenzie (Sep 09 2018 at 22:21):

And you're limited in doing it when the build is either not running or has progressed past a particular point.

Grahame Grieve (Sep 09 2018 at 22:21):

it's certainly an awful workflow. Astonishingly bad

Lloyd McKenzie (Sep 09 2018 at 22:23):

As an alternative, can we get access to (i.e. render) the pre-merge content on the CI build?

Grahame Grieve (Sep 09 2018 at 22:23):

that's an alternative how?

Lloyd McKenzie (Sep 09 2018 at 22:23):

That would offload the build process to the server.

Rob Hausam (Sep 09 2018 at 22:23):

I agree the build time is a concern - but I don't think there's a reason it can't work the same way Grahame does it now in svn (if I understand his process). My take is that you do some work on the "work branch" (could be only one or multiple of those) and then run the local build. While the build is running you can still work on some additional files (if that doesn't interfere with the build). Once the build is successful then the changes it reflects can be committed and pushed.

The big issue where I think there is a problem with the build delay, is that, as Lloyd says, you don't want to have "a bunch of files revert because you've switched branches and make the running build confused and either die or not reflect your changes." That actually will happen if you switch branches during a build and that has to be avoided. So that may push very strongly toward doing all of the work in a single "work branch" that multiple commits and pull requests are then generated from. I think that will end up being very similar to the workflow with svn - so maybe not actually "astonishingly bad"?

Lloyd McKenzie (Sep 09 2018 at 22:23):

You could run the build for your branch on the server, see if it looked good and if not, fix it

Grahame Grieve (Sep 09 2018 at 22:25):

@Josh Mandel consistently says "don't do another PR on a branch that's already merged"

Lloyd McKenzie (Sep 09 2018 at 22:25):

When you needed to make an interrupt, you'd stash your work, switch to a new branch, make your changes, commit and push them, then switch back and unstash. You wouldn't wait to test locally - your test would be the results of the server-side "ready to merge" build. We'd just need a flag for it to not auto-merge because you'd want to check that all was good with the output before merging.

Grahame Grieve (Sep 09 2018 at 22:26):

right now I'm trying remember what about GIT makes it actually better to justify all this awfulness

Lloyd McKenzie (Sep 09 2018 at 22:26):

Right. We wouldn't be doing that. You'd have your development branch and a patch-specific branch. You'd PR on the patch-specific branch.

Grahame Grieve (Sep 09 2018 at 22:26):

it seems like an incredibly high price to pay just to get checked builds

Lloyd McKenzie (Sep 09 2018 at 22:27):

Better = checked build, simpler typo correction, ability to add extra steps around "sensitive" (e.g. normative) files. Though it's possible that some of that could be done in SVN if we added hooks and stuff.

Grahame Grieve (Sep 09 2018 at 22:28):

simpler typo correction

Grahame Grieve (Sep 09 2018 at 22:28):

really?

Grahame Grieve (Sep 09 2018 at 22:28):

nothing about this is simpler at all

Lloyd McKenzie (Sep 09 2018 at 22:31):

typo correction people will be able to do from the Git website - no pushes/pulls etc.

Lloyd McKenzie (Sep 09 2018 at 22:31):

It's not simpler for change-proposal based stuff

Grahame Grieve (Sep 09 2018 at 22:33):

well, I've been meaning to comment on that. I'd actually rather not have that. It's me that processes the vast majority of the typo 'fixes'. only about 60% of them are valid and simple. about 10% are valid and real hard, and about 30% are not typos or not actually right or are html style issues

Grahame Grieve (Sep 09 2018 at 22:35):

and I always fix them using search/replace when I can, and about 50% of the time, I find the same mistake in more than one place

Rob Hausam (Sep 09 2018 at 22:43):

I don't think that committing new work and submitting a new pull request on a previously merged branch is actually a problem:
https://stackoverflow.com/questions/22010679/add-commits-to-a-merged-branch-and-start-a-new-pull-request

Rob Hausam (Sep 09 2018 at 22:43):

I suspect that is what you will want to do

Rob Hausam (Sep 09 2018 at 22:50):

It's also not appearing, though, that switching to git is making anything easier (other than typos) or solving any problems :(

Grahame Grieve (Sep 09 2018 at 22:55):

we do want the checked build

Grahame Grieve (Sep 09 2018 at 22:56):

have a look at this: https://github.com/HL7/fhir/pull/21

Grahame Grieve (Sep 09 2018 at 22:56):

why do we get all the past changes repeated?

Grahame Grieve (Sep 09 2018 at 23:32):

I tried following Lloyd's method. it failed, because of conflicting changes between branch and master. Scarily, the stash pop was partially successful, and left me with a trashed work space

Josh Mandel (Sep 09 2018 at 23:34):

Re: seeing changes repeated, is it possible you created a new branch off of your old branch, instead of from master?

Grahame Grieve (Sep 09 2018 at 23:35):

I suppose. But John got my changes in this pull:

Grahame Grieve (Sep 09 2018 at 23:35):

https://github.com/HL7/fhir/pull/20

Josh Mandel (Sep 09 2018 at 23:35):

When the branches are merged, I configured GitHub to do a "squash" , which means the commits themselves are squashed into a single commit beforehand merging.

Grahame Grieve (Sep 09 2018 at 23:36):

no he didn't. weird

Grahame Grieve (Sep 09 2018 at 23:39):

but I don't understand how your response relates to my question

Grahame Grieve (Sep 10 2018 at 00:32):

ok. my hate for git is increasing rapidly. this pull:

Grahame Grieve (Sep 10 2018 at 00:32):

https://github.com/HL7/fhir/pull/22

Grahame Grieve (Sep 10 2018 at 00:32):

it says: This branch has conflicts that must be resolved

Grahame Grieve (Sep 10 2018 at 00:33):

well, git, I did resolve the conflicts.

Grahame Grieve (Sep 10 2018 at 00:34):

but, wtf, I'll do what it says in how to resolve the conflicts. which, of course, is a spectacular fail. but eventually I pull/merge/edit 100s of conflicts until i have something that builds and that git says is ok. So then I commit to my local master.. because git says I don't know what I'm doing (and surely i have no clue at all) and it says I need to do that and not to change that. So I do. And then I switch to the branch in that pull.. and now git says that there is nothing to push, so.... I don't know how to fix that.

Grahame Grieve (Sep 10 2018 at 00:36):

but I know that I need to get this pushed so that I can get on with fixing the long list of other things people want me to fix today

Grahame Grieve (Sep 10 2018 at 00:36):

I guess the answer is, I'm too stupid to use git, so I don't know how I can get anything fixed

Josh Mandel (Sep 10 2018 at 00:40):

We can all be confident that the issue isn't with your intelligence. It's a learning curve to be sure,. But if you're encountering issues with the basics of making and merging isolated changes, it's probably an issue with my instructions.

Josh Mandel (Sep 10 2018 at 00:40):

Maybe we can set up a time for screen sharing / joint debugging to figure out what's going on.

Grahame Grieve (Sep 10 2018 at 00:41):

oh no I'm sure if I wiped everything and started afresh every time, it would be just dandy. But that completely is not how it's possible for me to work

Grahame Grieve (Sep 10 2018 at 00:41):

now would be really useful, since I'm stuck

Josh Mandel (Sep 10 2018 at 00:41):

Your definitely shouldn't ever have to "wipe everything".

Grahame Grieve (Sep 10 2018 at 00:41):

that's how your instructions work

Josh Mandel (Sep 10 2018 at 00:42):

Shouldn't be. Let's talk ; I'll message you in a minute when i get set up with a computer.

Grahame Grieve (Sep 10 2018 at 00:42):

k thx

Josh Mandel (Sep 10 2018 at 00:46):

https://hangouts.google.com/call/td_tS3u2_qIQtq1m14WNAAEE

Grahame Grieve (Sep 10 2018 at 01:20):

https://chat.fhir.org/#narrow/stream/138-committers.2Fgit-help/subject/Hello.20World/near/184483

Grahame Grieve (Sep 10 2018 at 02:43):

ok. long discussion with Josh (thanks). So some updates:

#1 how do you maintain a single local clone, but commit to multiple different branches

Josh has changed Github not to use squash merges. This means that we can commit to a branch after a PR on it. Also Josh very patiently demonstrated how switching works. The combination of how switching works and allowing commits to a branch after PR doesn't bring us back to the simplicity of SVN but it's probably workable. It's still much more mentally complex...

#2 how do we replace the build number system

The automated build numbering system is very difficult to replace, but the ways I use it to blow various caches downstream of the main build can be done just as well by introducing a manually controlled number somewhere. The pain is that I have to create a new place for that, add it to the package files, and then get all the downstream tools to be upgraded. But that's a once off pain

#3 how do we tell people how to know when a commit makes it's way onto build.fhir.org

Josh is going to update the build bot so that it adds the commit message to the build notification. Anyone to whom we used to say 'want for build X', the message will now be: watch the committers/notification for the commit message to appear. That's still incomplete because http caches might get in the way (this happens about 5% of the time) and there's no easy way to say 'you have the right version in practice' (user will have to scan the history of the build bot and see what hash they're getting. difficult when we're really busy)

#4 how to auto-merge commits from authorised users

Josh is going to look at mergify for this

#5 how to auto-delete local branches that are dead

we didn't discuss this

#6 (new) do we have to have topics for each branch

we didn't discuss this

Lloyd McKenzie (Sep 10 2018 at 02:46):

Will there be some improved documentation forthcoming around #1 (and changes in the documentation in general to not presume the use of commandline)?

Josh Mandel (Sep 10 2018 at 02:47):

Re: docs, if we can define patterns that work for committers in the community, let's by all means document them. I think we're a little bit today still at the stage of throwing things at the wall to see what sticks.

Josh Mandel (Sep 10 2018 at 02:47):

Re: CLI, I'd love if there are volunteers to help with platform-specific GUI tool advice + docs. Again, I think we need a well-defined workflow, and working CLI instructions, before we attempt to "port" these as instructions for platform-specific GUI tools.

Bryn Rhodes (Sep 10 2018 at 02:50):

Agreed and happy to help with the Windows/TortoiseGit instructions

Grahame Grieve (Sep 10 2018 at 03:27):

ok another question. I switched to my local master, and it said my local master was 6 commits ahead of the remote master... I can't think how that makes sense. So I did a pull, and now it's 7 commits ahead of the remote master.

Grahame Grieve (Sep 10 2018 at 03:27):

that doesn't make sense to me

Grahame Grieve (Sep 10 2018 at 06:10):

and also, @Josh Mandel I very definitely derived the branch used here from my master: https://github.com/HL7/fhir/pull/23

Grahame Grieve (Sep 10 2018 at 06:11):

but it still claims to be based further back that it should have been, which is only the last change. Presumably, I have somehow been unable to get my master actually current?

Grahame Grieve (Sep 10 2018 at 06:11):

(since it only contains the actual changes I definitely wanted)

Joel Schneider (Sep 10 2018 at 14:50):

Here's an example of a Git branching model, known as "GitFlow", that may be of interest.
https://nvie.com/posts/a-successful-git-branching-model/
Being new to the FHIR & Git discussion, I'm unsure how well this aligns with what has already been established, but it may be worth a look.

Vadim Peretokin (Sep 11 2018 at 12:36):

ok another question. I switched to my local master, and it said my local master was 6 commits ahead of the remote master...

Remember when you commit, unlike SVN, it doesn't automatically go up to the server. So you made 6 commits locally and didn't push them to remote

So I did a pull, and now it's 7 commits ahead of the remote master.

Maybe you got the latest commits from remote and merged them (so merge-commit, 1 commit) hence why you're 7 ahead now. Push and you won't be ahead anymore

Josh Mandel (Sep 11 2018 at 12:54):

@Vadim Peretokin keep in mind that the master branch on GitHub.com/hl7/fhir can't be pushed to directly -- only gets updated via PR.

Vadim Peretokin (Sep 11 2018 at 12:57):

Right, sorry (and that's very good!). Making a PR for those and merging that PR would then get your local master up to date.

Though in my workflow I never bother with the local master - I just make a new branch from the remote master for every logical piece of work that I do.

Josh Mandel (Sep 11 2018 at 12:59):

Do you want to jot down the way you get a new feature branch started, so we can we exactly what this means?

Vadim Peretokin (Sep 11 2018 at 13:11):

I'll have a look at that.

Geoff Low (Sep 11 2018 at 15:27):

Feature branches represent a set of tasks; something motivates GG to (for example) fix a set of typos - it may be a forge issue or similar; you create a feature branch for that set of work. We have jira issues representing both Bug fixes and New Features, and the unique identifiers for them are usually included in the feature branch name. The idea is you commit as you go, so you don't have unstaged changes; stashing is a PITA in my experience.

All those tasks to close out the issue are bundled together in the feature branch and when the work is complete, the branch is then pushed up for a PR merge the changes in the current HEAD branch;

Conventionally, in the git-flow model, the develop branch would be the HEAD branch and represents the current working state (so would be the source for build.fhir.org for example); master represents the last released version (and there are release branches for past releases)

Josh Mandel (Sep 11 2018 at 15:37):

This is a great overview! I should note for the record that we are not proposing to follow the git-flow model right now; we are doing something that looks a lot more like the familiar subversion workflow to try to minimize the pain of transition -- that is to say, a single master branch that represents the current state of the spec (generally in flux, but not broken), with release tags from time to time as we publish releases.

Geoff Low (Sep 11 2018 at 20:05):

yeah, we follow it 'loosely' - when you start talking aboutrelease and hotfix branches people (who've been doing this much longer than I have) get confused - as do I.

Geoff Low (Sep 11 2018 at 20:09):

that said, I do find the distinction between master and develop to be helpful;

Last updated: Apr 12 2022 at 19:14 UTC