FHIR Chat · Transition to GitHub · committers

Stream: committers

Topic: Transition to GitHub

Josh Mandel (Sep 04 2018 at 21:19):

To @all FHIR Committers: prior to the September Connectathon+WGM, we're undertaking a migration of the FHIR source repository from GForge/subversion to GitHub. We expect this this transition will provide advantages for maintainability, including the ability to run automatic build status checks and request review and sign-off before merging code — as well as a much lower barrier for proposing small tweaks like spelling corrections. At the same time, we know there's a learning curve associated with git and with GitHub, so we're setting up documentation plus a team of helpers on Zulip (#committers/git-help) to ease the transition.

To get started, please read "Get Started with FHIR on GitHub" (preview here) and fire off any questions here so we can start developing a FAQ and supporting docs :-)

Oliver Egger (Sep 05 2018 at 19:42):

short question about non- (or not yet) committers, will there be also a process supported for submitting patches trough a fork/pull workflow like described in https://gist.github.com/Chaser324/ce0505fbed06b947d962 ?

Lloyd McKenzie (Sep 05 2018 at 19:59):

I think we'd like to get there. We'll probably need to tie that into Jira so we've got a single point of traceability for change submissions. (So likely a "phase 2" thing.)

Josh Mandel (Sep 05 2018 at 20:00):

Yes -- any committer will be able to shepherd an external PR.

Josh Mandel (Sep 05 2018 at 20:01):

That's the technical answer. In terms of policy, we will have to develop guidance about when this is good/expected (e.g., anyone contributing content may need to agree to a contributor license before their content can be incorporated).

Josh Mandel (Sep 05 2018 at 20:01):

So far we've focused on supporting our existing svn contribution model.

Josh Mandel (Sep 07 2018 at 17:56):

Okay -- we've made the cutover! http://github.com/hl7/fhir is live and ready for your PRs (assuming you follow https://github.com/HL7/fhir/wiki/Get-Started-with-FHIR-on-GitHub)

Jason Walonoski (Sep 07 2018 at 19:51):

You should change the project description Mirrors the FHIR SVN repo (via git-svn) since it isn't exactly true anymore.

Jason Walonoski (Sep 07 2018 at 20:30):

Also, the build is still trying the read the SVN number from the repo. So that will need to be fixed.

Jason Walonoski (Sep 07 2018 at 20:31):

     [java] Warning @ Unable to read the SVN version number: svn: E155007: '/Users/jwalonoski/git/fhir' is not a working copy

Grahame Grieve (Sep 07 2018 at 20:47):

I'm not sure what to do about that. Under svn, the build number allows you to reconstruct the source used to produce the build. We need an equivalent in git

Jason Walonoski (Sep 07 2018 at 20:54):

I humbly suggest that you could use the git commit number. In Synthea we shell out the command git describe --tags --always which returns the commit hash, and you can use that as your build number.

Josh Mandel (Sep 07 2018 at 21:10):

I'm using the commit hash (--short) on the CI build already

Josh Mandel (Sep 07 2018 at 21:11):

See the bottom line of http://build.fhir.org

Josh Mandel (Sep 07 2018 at 21:12):

This allows rebuilding by anyone with the source, very straightforward as Jay suggests.

Josh Mandel (Sep 07 2018 at 21:13):

I need to do the same for the PR builds (though they're not published anywhere for the moment, so it doesn't matter yet).

Grahame Grieve (Sep 07 2018 at 21:15):

what did you change for the CI Build?

Josh Mandel (Sep 07 2018 at 21:15):

Using the git commit hash as the version argument to the publish script, so it appears in the output

Josh Mandel (Sep 07 2018 at 21:16):

(this is just in the VSTS config, so not visible in GitHub directly right now.)

Grahame Grieve (Sep 07 2018 at 21:16):

I'm really really going to miss a linear build number. I think it's a genuinely big issue. I can't tell you how much the derivative eco-system depends on build number.

Grahame Grieve (Sep 07 2018 at 21:16):

can we make the PR system define a serially incrementing number on the trunk?

Josh Mandel (Sep 07 2018 at 21:17):

We should be tagging with numbers whenever we want this kind of behavior.

Grahame Grieve (Sep 07 2018 at 21:17):

we want this behavior on every commit to the trunk, and then to be picked up on every branch derived from the trunk

Josh Mandel (Sep 07 2018 at 21:17):

Not easily; I don't think we should try this, since there will be ways to slip through the cracks.

Grahame Grieve (Sep 07 2018 at 21:18):

well, easy to say for you, who aren't so harmed by the lack of it. It's a big deal for me

Josh Mandel (Sep 07 2018 at 21:20):

I'm just saying I've never seen this done in Git, and I'm wary of trying to invent a new strategy here. Does anyone here have experience with this?

Grahame Grieve (Sep 07 2018 at 21:20):

and you must have made some changes in the java code for the CI build? but I bet you just broke all the rest of the build, the non-visual parts... what change did you make?

Josh Mandel (Sep 07 2018 at 21:21):

There was a one line change -- one sec

Josh Mandel (Sep 07 2018 at 21:22):

Never mind -- https://github.com/HL7/fhir/pull/14 wasn't merged yet. So, not sure what you're referring to, but "git diff" will show you recent changes

Josh Mandel (Sep 07 2018 at 21:23):

And "git log -p" will show you all commits with code changes inline.

Grahame Grieve (Sep 07 2018 at 21:23):

I

Grahame Grieve (Sep 07 2018 at 21:24):

I'm rapidly falling back to the hate git thing again. I don't know how I'm going to get by without the serially incrementing number. I already have enough trouble trying to keep things together

Grahame Grieve (Sep 07 2018 at 21:24):

and I hate the command line

Lloyd McKenzie (Sep 07 2018 at 21:25):

I'm assuming we don't have to use the commandline - that we can use tortoisegit the way we used svn. That's what I did with IGs - though I guess I never needed to do pull requests there...

Grahame Grieve (Sep 07 2018 at 21:25):

all the documentation is command line based, and I think that for some of it, that's the only way to do things

Josh Mandel (Sep 07 2018 at 21:26):

Well, please keep the questions flowing on #committers/git-help

Josh Mandel (Sep 07 2018 at 21:27):

We'll do our best to push over the pain of the transition.

Grahame Grieve (Sep 07 2018 at 21:27):

right now I'm not feeling that 'suck it up' is the best help with regard to the build number

Grahame Grieve (Sep 07 2018 at 21:28):

we have downstream synchronization code that depends on knowing whether the build number is more recent than a given fixed value

Josh Mandel (Sep 07 2018 at 21:28):

I'm open to ideas; tagging is the closest thing I know to what you're asking about.

Josh Mandel (Sep 07 2018 at 21:28):

That's a very common approach for marking explicit releases.

Josh Mandel (Sep 07 2018 at 21:29):

Otherwise, we can point to code that tells you whether one revision comes before another...

Grahame Grieve (Sep 07 2018 at 21:30):

code that tells you whether one revision comes before another

would have to go look in the git history?

Josh Mandel (Sep 07 2018 at 21:30):

https://stackoverflow.com/a/13526591

Josh Mandel (Sep 07 2018 at 21:30):

Right

Josh Mandel (Sep 07 2018 at 21:32):

How does the current code work? How does it know the svn revision of the build it's working with?

Grahame Grieve (Sep 07 2018 at 21:32):

the current code looks it up from svn

Josh Mandel (Sep 07 2018 at 21:33):

Okay, so it has access to the repository

Josh Mandel (Sep 07 2018 at 21:33):

Okay, so it has access to the repository

Grahame Grieve (Sep 07 2018 at 21:33):

what we're saying is that we're going from a system where you could just know how recent the base of the code was, to a system where you can only establish this by looking it up in git

Josh Mandel (Sep 07 2018 at 21:33):

Ok, so it has access to the repository, similar to what you'd need with git.

Grahame Grieve (Sep 07 2018 at 21:34):

the build always runs in the context of a version control system. if you tried to remove it from an svn context, it would fail.

Grahame Grieve (Sep 07 2018 at 21:34):

well, actually, the build would work, but anything downstream would fail. I will start getting failure reports pretty soon, acually

Jason Walonoski (Sep 07 2018 at 21:34):

Apparently, this is controversial, but I submitted a pull request https://github.com/HL7/fhir/pull/17 to use the hash.

Jason Walonoski (Sep 07 2018 at 21:34):

We can sort of fix this...

Jason Walonoski (Sep 07 2018 at 21:35):

git rev-list --count HEAD

Jason Walonoski (Sep 07 2018 at 21:35):

That will tell you the sequence number of the last commit.

Josh Mandel (Sep 07 2018 at 21:35):

Cool -- what's a sequence number?

Jason Walonoski (Sep 07 2018 at 21:36):

The pull request I submitted will output something like Version 3.5.0-4b6f35af3b, but using the above command it would become Version 3.5.0-11949

Josh Mandel (Sep 07 2018 at 21:36):

And does this change or break with shallow clones?

Jason Walonoski (Sep 07 2018 at 21:37):

It is literally the length of the commit log. Regarding breaks -- don't know to be honest. Never had to deal with this before.

Grahame Grieve (Sep 07 2018 at 21:38):

well, I'm screwed either way.

Josh Mandel (Sep 07 2018 at 21:38):

I'd be careful with this. I worry it's going to be a bit context dependent, but we should explore.

Josh Mandel (Sep 07 2018 at 21:38):

Hey, are you saying you want to switch back?

Josh Mandel (Sep 07 2018 at 21:38):

We're only 5h into this; easy enough to reverse.

Grahame Grieve (Sep 07 2018 at 21:39):

I don't want to reverse. You've done a lot of work, and we have to make the transition.

Grahame Grieve (Sep 07 2018 at 21:39):

but I don't know how I'm going to deal with this going forward

Jason Walonoski (Sep 07 2018 at 21:39):

I think it changes depending on the branch. But so long as official builds are always built from master it should be fine. https://stackoverflow.com/questions/4120001/what-is-the-git-equivalent-for-revision-number

Grahame Grieve (Sep 07 2018 at 21:40):

so we can't get the PR build to do anything to generate something into the commit that follows the PR build?

Josh Mandel (Sep 07 2018 at 21:42):

Jay, if you do a shallow clone, does the count go down? (I'd try but I'm on mobile right now.(

Josh Mandel (Sep 07 2018 at 21:43):

You could try to, like, write a file -- but there's race conditions there.

Josh Mandel (Sep 07 2018 at 21:44):

We could automatically tag each new commit with an increasing number

Grahame Grieve (Sep 07 2018 at 21:44):

I can think about the race conditions. but how would we write a file?

Grahame Grieve (Sep 07 2018 at 21:44):

We could automatically tag each new commit with an increasing number

Josh Mandel (Sep 07 2018 at 21:44):

Amending each PR before merging (yuck).

Grahame Grieve (Sep 07 2018 at 21:44):

how would we do that? commits to trunk?

Josh Mandel (Sep 07 2018 at 21:44):

But if your key question is "does A come before B," I want to understand why git ancestor tests don't work.

Grahame Grieve (Sep 07 2018 at 21:45):

because in the context where I do the test, I don't have access to git, or even necessarily the web

Josh Mandel (Sep 07 2018 at 21:45):

You'd try something like a script to modify every PR branch before merging it. (I definitely don't like this idea.)

Josh Mandel (Sep 07 2018 at 21:45):

You don't need the web...

Grahame Grieve (Sep 07 2018 at 21:45):

several key places in derivative tools checking for currency

Josh Mandel (Sep 07 2018 at 21:46):

Can you share more about the context?

Jason Walonoski (Sep 07 2018 at 21:46):

Shallow clone messes up the rev-list result.

Josh Mandel (Sep 07 2018 at 21:46):

Okay, thanks Jay! It's not necessarily terrible, but important to know what we're dealing with.

Grahame Grieve (Sep 07 2018 at 21:50):

the technical output from the build is marked with the build version. As is the same output from the local build. it goes in each page, it goes in the IGpublisher, and it goes in the packages that get produced.

It's common discussion for me to have with people: what version of (build.fhir.org) are you using, what version of the ig publisher are you using. we would say 13800 or more recent.

Then, there's a file http://build.fhir.org/package-min-ver.json that tells all my tooling what's the minimum acceptable build version for any packages/caches linked to the current build. if I set that to the next version when committing, then all of the packages/caches will be wiped (that one does depend on web access, people off the web won't pick up the fix anyway)

I'm not sure that's the full list....

Grahame Grieve (Sep 07 2018 at 21:51):

how do you think that that currency discussion will work?

Grahame Grieve (Sep 07 2018 at 21:51):

it's possible, given that the package-min-ver system requires web access anyway, it's possible I could just work around it

Josh Mandel (Sep 07 2018 at 21:54):

if I set that to the next version when committing, then

This is the crux of it. With git and PRs, you can't know what the next revisions will be; someone else could merge first. (That said, in think the same is true for svn...)

Josh Mandel (Sep 07 2018 at 21:55):

Hmm.

Grahame Grieve (Sep 07 2018 at 21:55):

yes there's a risk of that. it hasn't been a problem in practice

Josh Mandel (Sep 07 2018 at 21:57):

I mean, you could certainly Express something like "expire any caches associated with commits equal to or earlier than d472be2a" (where that's the current master), even without knowing what the next is.

Grahame Grieve (Sep 07 2018 at 21:57):

how can I express that?

Josh Mandel (Sep 07 2018 at 21:57):

In the semantics of your min-ver file.

Josh Mandel (Sep 07 2018 at 21:58):

You'd have a property like "must-be-newer-than".

Josh Mandel (Sep 07 2018 at 21:58):

Otherwise, I think explicit tags is best

Grahame Grieve (Sep 07 2018 at 21:58):

alright, I asked the wrong question. I'm a tool, I've got a hash code from min-ver. I want to know whether some other hash code is newer or older than the hash code from min-ver...

Grahame Grieve (Sep 07 2018 at 21:58):

how?

Josh Mandel (Sep 07 2018 at 21:59):

Especially if you don't need to reference every single commit this way

Josh Mandel (Sep 07 2018 at 21:59):

The hash comparison only works if you have a git repo with history.

Josh Mandel (Sep 07 2018 at 21:59):

Otherwise, this is really what tags are for

Josh Mandel (Sep 07 2018 at 22:00):

When you're making changes, you can explicitly define a tag that you want to apply to the commit on master, and then when your changes land on master, you apply the tag.

Josh Mandel (Sep 07 2018 at 22:01):

The tags can be simple integers, or follow whatever pattern you like.

Grahame Grieve (Sep 07 2018 at 22:01):

as for humans, here's a super common work flow for - probably averages 3 times week, but as publication approaches, can be 1/hr on average:

a. user X reports an issue to me. They're hanging out for a fix
b. I figure out the issue, commit something. I get a commit number
c. I tell them to wait until the commit number is built (by checking build.fhir.org)
d. once it clicks over, they're fixed and good to go

I think that the work flow for this is now... absent

Josh Mandel (Sep 07 2018 at 22:03):

(You could point them to the PR that fixed the issue and they could get the commit hashes etc from there, but agreed it's more work.)

Grahame Grieve (Sep 07 2018 at 22:04):

but they won't know when it's built anyway, because how will they know when the build is derived from the trunk that has that PR?

Josh Mandel (Sep 07 2018 at 22:05):

The PR will show the merge. The merge commit hash will appear on build.fhir.org

Grahame Grieve (Sep 07 2018 at 22:05):

transiently. some other one will probably overwrite it a little later. then what?

Grahame Grieve (Sep 07 2018 at 22:06):

I'm sorry, but this seems so fundamental to me. how can this be hard?

Josh Mandel (Sep 07 2018 at 22:08):

I think it's the price for branching and merging to be easy; you lose linearity.

Grahame Grieve (Sep 07 2018 at 22:08):

I thought we added *.class to gitIgnore

Grahame Grieve (Sep 07 2018 at 22:08):

I'm not asking for linearity outside the head

Grahame Grieve (Sep 07 2018 at 22:09):

presumably I'm just stupid because I think that there's workflow associated with the linearity of the head, which does exist.

Josh Mandel (Sep 07 2018 at 22:09):

I thought we did too. Just merged https://github.com/HL7/fhir/pull/15 now.

Grahame Grieve (Sep 07 2018 at 22:09):

to think that people actually care

Josh Mandel (Sep 07 2018 at 22:10):

FWIW, I set up GitHub to ensure that master stays linear; it's a protected branch and only allows "squash" merges, so each PR becomes one single commit on master.

Grahame Grieve (Sep 07 2018 at 22:11):

there's some other option?

Grahame Grieve (Sep 07 2018 at 22:11):

I was about to ask what squash meant

Josh Mandel (Sep 07 2018 at 22:11):

It's an option in the GH settings, to limit the types of merges allowed.

Josh Mandel (Sep 07 2018 at 22:12):

Squash keeps things linear, and compresses multiple commits into one when merging a PR.

Grahame Grieve (Sep 07 2018 at 23:31):

another issue - I suppose we are using a standard git/zulip bot. is it really value to have a topic per branch? (when we have a branch per pull request). that's also horrible. Is that a Zulip issue?

Lloyd McKenzie (Sep 08 2018 at 00:45):

It makes sense when pull requests are going to get reviewed and might be commented on. It doesn't make much sense when they're auto-committed on success (and failure is noted elsewhere)

Grahame Grieve (Sep 08 2018 at 00:46):

we're not auto-committing on success. that's not even in the plan.

Grahame Grieve (Sep 08 2018 at 00:46):

it's just part of the increased workload for using git - a lot more than i expected.

Lloyd McKenzie (Sep 08 2018 at 00:51):

Why would we not auto-commit on success if it's possible? Once we have content that requires approvals, sure. But until then, it's a needless no-value step.

Lloyd McKenzie (Sep 08 2018 at 00:52):

You've both committed and pushed. Both are clear signals that you want it in the master. What's the value of a third confirmation that "yes, I really really really want this in the master"?

Grahame Grieve (Sep 08 2018 at 00:53):

you are talking about commits by users authorised to approve pull requests.

Grahame Grieve (Sep 08 2018 at 00:54):

I identified this as an issue earlier, but Josh said it wasn't possible? normal?

Grahame Grieve (Sep 08 2018 at 00:54):

it just seems like yet another way to have an inefficient workflow to me.

Lloyd McKenzie (Sep 08 2018 at 00:56):

So the intention is that someone is reviewing all commits now?

Grahame Grieve (Sep 08 2018 at 00:56):

I used to review all commits. It stopped being possible a long time ago

Lloyd McKenzie (Sep 08 2018 at 00:57):

Right. So we're not reviewing all commits?

Lloyd McKenzie (Sep 08 2018 at 00:57):

Because if we're not reviewing them, then anyone who was a committer on gForge should be able to auto-push to master

Lloyd McKenzie (Sep 08 2018 at 00:58):

Obviously that can't fly for commits from the public which we'll have to review.

Grahame Grieve (Sep 08 2018 at 00:58):

here's a super common work flow for me:
- user reports an issue
- I make a fix and go on with what I'm doing
- I debug the fix next time I run the build for whatever I'm doing
- I confirm that the fix works
- I commit the fix

now that we've moved to git, I think the work flow becomes:
- create a new local clone with a named branch
- open it as a new workarea in eclipse
- configure the output from the work area
- make the fix
- run under the debugger & confirm fixed.
- don't build on the other workarea (computer can't take 2 builds)
- commit/pullrequest/approve pull request

I can't do this. It has to be easier. I'm tapped to the limit, and it's already a problem...

Lloyd McKenzie (Sep 08 2018 at 00:59):

Can you commit something to a branch that's not currently open?

Lloyd McKenzie (Sep 08 2018 at 01:00):

It may be better to just have "branch of stuff I'm doing today" and commit to that and push when ready. Saves switching branches.

Grahame Grieve (Sep 08 2018 at 01:02):

but I don't work like that. the stuff I'm doing today is typically an interlacing of
- a big project i'm carrying forward (often take many days)
- small things I'm rolling over as I have a chance to look at them
- urgent fixes for people

Grahame Grieve (Sep 08 2018 at 01:03):

I was going to try "just have "branch of stuff I'm doing today" and commit to that and push when ready" but josh says not to use a branch after a merge from it... so that's not really possible

Lloyd McKenzie (Sep 08 2018 at 01:03):

So the issue for you is you'll have a whole bunch of uncommitted stuff while you want to make a commit and a push, but really don't want to switch branches.

Lloyd McKenzie (Sep 08 2018 at 01:04):

If you could "commit to branch X", that could work for you. Though this is probably a better discussion to have on the #committers/git-help thread where people are supposed to be monitoring who can answer whether that's possible...

Grahame Grieve (Sep 08 2018 at 01:47):

I don't mind switching branches, but what I'm undestanding right now is that it isn't really possible to a clone locally and just commit bits of it to whatever branch I want.

Rob Hausam (Sep 08 2018 at 05:27):

You can create and commit to as many local branches as you want. And then push those separate branches with their commits to origin (your clone), and generate pull requests and merge them to master whenever ready. You should be able to work it out so you can keep working on the big project branches for as long as you need to, and open, push and close the smaller branches separately in parallel with that. I think that might be mostly what you want and need - but there may be more.

Grahame Grieve (Sep 08 2018 at 21:58):

what it appears to me is that I can only do that from different clones on my local drive. if I can do that from a single clone... please explain

Grahame Grieve (Sep 08 2018 at 22:05):

back to the build version thing: it's possible that I can replace the linear build version thing in the code with simply updating the fhir version (e.g. 3.6.0 --> 3.6.1). It's possible I should do it that way. But it takes me about 90min to change the version, instead of 20secs with the old way. I think that's pretty significant. It's possible I could reduce that, with practice - it happens rarely now, partly because it takes so long - but it does have a long sting in the tail for me.

But whatever, the fact that we don't have a linearly increasing build version will cost me a lot of time. And I still have no idea how I can tell someone 'I committed a fix, when x happens, the fix is available to you'

Josh Mandel (Sep 09 2018 at 02:55):

You only need one clone; you can switch quickly back and forth between branches by using the "git checkout" command. This is perhaps the quintessential feature of git.

Josh Mandel (Sep 09 2018 at 02:57):

Can you say more about where the 90 minutes comes from? If you're willing to manually update something in the code base, then using that manual effort to update something as simple as a text file with a version number might be able to achieve the same thing.

Lloyd McKenzie (Sep 09 2018 at 03:26):

The use-case Grahame has described is that he wants to be able to work in one directory (and build against that one directory) without switching branches while committing and pushing subsets of the content from that directory. Is that possible? I do the same thing, though not nearly with the frequency Grahame does.

Josh Mandel (Sep 09 2018 at 03:28):

I didn't follow the exact use case. If the question is "can I push subsets of updated content," then sure. A given commit only has whatever you git add to put in it. So if you've changed 10 files but only want to push up 3 of them, you can git add f1.txt f2.txt f3.txt and then git commit and the git push.

Lloyd McKenzie (Sep 09 2018 at 03:35):

This is more commonly changed files rather than added ones. The issue is the prohibition against multiple pushes of the same branch

Lloyd McKenzie (Sep 09 2018 at 03:35):

How can I have a bunch of changed files in my folder, commit and push a subset, keep working, commit and push a different subset, etc.

Josh Mandel (Sep 09 2018 at 03:50):

What do you mean, "prohibition against multiple pushes of the same branch"?

Josh Mandel (Sep 09 2018 at 03:50):

You can just do exactly as you've said: commit and push, commit and push,

Josh Mandel (Sep 09 2018 at 03:51):

It looks like:

1. Change 10 files
2. Commit the first few via git add f1 f2 f3; git commit -m "some initial changes"; git push
3. Later repeat with f4...f10.

Lloyd McKenzie (Sep 09 2018 at 04:01):

But I thought you said we weren't supposed to push multiple times with the same branch?

Lloyd McKenzie (Sep 09 2018 at 04:01):

Because what gets pushed should get merged right away

Josh Mandel (Sep 09 2018 at 04:11):

What I tried to say was: once you merge a branch into master, don't keep pushing to it.

Josh Mandel (Sep 09 2018 at 04:11):

Before you merge it, you should free free to organize your workflow in whatever manner suits you.

Lloyd McKenzie (Sep 09 2018 at 04:38):

Right. And what Grahame wants to do is commit some stuff and merge it, then commit more stuff and merge it, etc. Without switching branches

Lloyd McKenzie (Sep 09 2018 at 04:39):

Can you commit content to a branch you're not in?

Lloyd McKenzie (Sep 09 2018 at 04:40):

Or alternatively can you move where you currently are to being a different branch while having uncommitted content?

Lloyd McKenzie (Sep 09 2018 at 04:41):

E.g. Work in a "working" branch that's never committed. Then decide "oh, I want to commit some stuff", so clone/move to a new branch, commit a few things, push and (auto)merge. Then work some more. Then clone/move to a new branch, commit a few things, push and (auto)merge

Rob Hausam (Sep 09 2018 at 04:45):

yes, if you stash the changed content before you switch to a different branch
then you can pop or apply the stashed changes when you are ready to come back and continue working on them

Josh Mandel (Sep 09 2018 at 04:45):

I'm having trouble following. In general, you shouldn't be working with more than one local clone at a time.

Josh Mandel (Sep 09 2018 at 04:46):

And yes, as Rob says: between stashing (git stash) and checking out various branches it's pretty easy to jump around.

Josh Mandel (Sep 09 2018 at 04:47):

For getting to know git though, I'd highly recommend committing (even without pushing) rather than stashing, to make sure you have an explicit handle on the work you've done so it's easier to pick up later.

Rob Hausam (Sep 09 2018 at 04:50):

agree with Josh - but there might be some times where you would want to stash rather than commit (as discussed in the Git documentation) - and you have the ability to do either one

Rob Hausam (Sep 09 2018 at 04:53):

I'm generally inclined to commit, too - I've only rarely used stash (so far)

Lloyd McKenzie (Sep 09 2018 at 05:01):

The key thing is that Grahame (and others) may be working on multiple problems at once. So they're not working in a problem-specific branch. They're working in mainline or a "get stuff done" branch that's never committed. Given that, how do you commit and push multiple branches?

Lloyd McKenzie (Sep 09 2018 at 05:02):

You want to be able to build with all of the changes you're working on, but push and merge them separately

Grahame Grieve (Sep 09 2018 at 11:02):

I have a local clone. It's really hard for me to create another local clone because I have all sorts of tools, bookmarks, etc tied to the local location of that clone. (which is c:\work\org.hl7.fhir\build, a path everyone will have seen at some time or other). I typically have multiple things going on in that workspace. Under svn, I just committed the things I wanted to commit when I wanted to commit.

What I haven't figured out with Git is: how do I do this? It seems that my clone is tied to a single branch, and I can't commit to another branch locally without some... operation.. of some sort that might or might not trash what I'm doing.

When @Ewout Kramer sold me on it, it was because I'd get more control over how to commit my work, but right now it seems that I have a lot less

Grahame Grieve (Sep 09 2018 at 11:09):

@Josh Mandel updating the version number takes 90min - more, actually - because of all the things that have to be done in sync with each other when I change the version number.

Grahame Grieve (Sep 09 2018 at 11:09):

here's the instructions:

Grahame Grieve (Sep 09 2018 at 11:11):

; see the version policy in versions.html, and consult the FHIR Product Director before changing this.
1# update, make sure you have the latest copy, and make sure that the java reference implementation is synced
2#. do a successful build
3# make the version change, and make an entry in build\source\history.html
3a# and replace any occurances of he old version in the xml files in \source and \guide unless they are specifically referring to a past version (e.g. not in history.html)
4#. start a build
5#. when the build fails, copy the java reference to the tools (use \build\tools\java\compare-current-generated-code.bat)
6# run the build again
7# now upgrade the terminology server for the new build, and run it locally
8# now delete the terminology cache, and run the build again against the local terminology server
9# update the version of tools\html\package-min-ver.json to force the current fhir package to be reloaded everywhere
10# now commit everything

Grahame Grieve (Sep 09 2018 at 11:12):

it's possible that now the conformance resources are finally frozen, I can reduce the length of this list. I'm not sure right now, and now is not a time for things to blow up with regard to this

Josh Mandel (Sep 09 2018 at 12:43):

Thanks for spelling that out -- I'm not sure where to go with the version update story, because I don't really know about most of these pieces. Does anyone else (besides you Grahame) ever follow these steps?

Josh Mandel (Sep 09 2018 at 12:49):

Re: branches, git is very flexible here. But blindly committing a change to a branch you don't have open is... not making a ton of sense to me. Can you give an example of the kind of thing you have in mind?

Rob Hausam (Sep 09 2018 at 14:32):

@Grahame Grieve The only operation you need for switching to commit to different branches of your single local clone is 'git checkout', in combination with committing first before you checkout and switch to a different branch or using 'git stash' to save the state of your work (including 'git stash -u' or 'git stash -a' if you also want to save the state of untracked or even "ignored" files).

Lloyd McKenzie (Sep 09 2018 at 15:28):

The workflow in SVN is I have a single working folder. I can be working on many problems at once in that working folder and when I build, I build with all of the changes made. I can then commit subsets of them at will to the master. With Git, the "preferred" approach is that you solve each problem in a separate branch and do a local gen, commit and push separately for each branch. But the problem with that is the local gen. It can take anywhere from 30-45 minutes. And while it's running, you can't mess around switching branches. The result is that the idea of one problem, one branch is a non-starter. But there's still a need to commit and push incrementally. And yes, that means that you're committing content where you haven't done a local build where that content is the only thing that's change. For some people, that's dangerous. Even Grahame will occasionally mess it up. But that's what the CI Build process is for. If the local build took 2 minutes, we wouldn't be having this conversation.

Josh Mandel (Sep 09 2018 at 16:46):

So the svn process isn't "safe" in the sense that you're pushing subsets of a complete change set that haven't been individually tested. If you're willing to do this, then pushing untested code to a GitHub branch should be strictly better (since at least tests will run remotely). In other words, if you're comfortable skipping the local build then you don't take the 30 minute hit.

Lloyd McKenzie (Sep 09 2018 at 18:47):

What's the process for making that scenario happen? I.e. how can I work in a folder with multiple changes and commit/push/auto-merge a subset of those changes?

Josh Mandel (Sep 09 2018 at 21:15):

From above...

Josh Mandel (Sep 09 2018 at 21:15):

It looks like:

1. Change 10 files
2. Commit the first few via git add f1 f2 f3; git commit -m "some initial changes"; git push
3. Later repeat with f4...f10.

Grahame Grieve (Sep 09 2018 at 21:15):

where's the branch switch?

Lloyd McKenzie (Sep 09 2018 at 21:20):

Note that there has to be a merge right after the git push - the whole point of the commit is to get the content into the CI build.

Josh Mandel (Sep 09 2018 at 21:39):

We're going in circles, probably because I don't understand the use case yet. But once you merge, you don't want to keep pushing commits to a remote branch

Grahame Grieve (Sep 09 2018 at 21:43):

you don't want to keep pushing commits to a remote branch

but your advice was to repeat with f4..f10. We are going in circles, but the use case seems pretty basic to me: I am doing different things at the same time, and I want to commit one cause it's done (and other people are waiting for it) while I continue working on the other. In all the time I've been working with version control, this is basic workflow # 2

Bryn Rhodes (Sep 10 2018 at 01:02):

I use stash to save out my local changes, create the branch to commit, stash pop, commit only what I want to on that branch, then stash the rest again to switch back to master and stash pop.

Grahame Grieve (Sep 10 2018 at 02:49):

the thing that's problematic for me is that until you commit something, it's in your local space and remains when you switch branches. but once you commit it, then it disappears when you switch

Rob Hausam (Sep 10 2018 at 04:00):

What do you mean "disappears when you switch"? What "disappears"? I'm not following that.

Grahame Grieve (Sep 10 2018 at 04:08):

if you have an uncommitted change in your folder, when you checkout a different branch, it will stay in your folder. But once you commit it, when you change branches to a branch that doesn't have that change, the change will disappear

Lloyd McKenzie (Sep 10 2018 at 04:19):

I guess you'd have to merge the branch you committed it from into your "main" branch

Rob Hausam (Sep 10 2018 at 06:11):

yes, in that case you'll need to merge the branch you committed from into the newly checked out one
and you can cherry-pick individual commits, if needed

Josh Mandel (Sep 12 2018 at 23:25):

I added _JAVA_OPTIONS=-Xmx3200m as an env variable in our build system to try to keep 4 builds within 16Gb of RAM. Checking now to see if we can get 4 builds to run in parallel.

Josh Mandel (Sep 12 2018 at 23:26):

Hmm @Grahame Grieve not a RAM issue but one build failed like this: https://fhir-build.visualstudio.com/build.fhir.org/_build/results?buildId=719&view=logs -- could multiple build be clobbering each other's temp space? Do I need to isolate them?

Josh Mandel (Sep 12 2018 at 23:26):

2018-09-12T23:24:18.2095662Z      [java] FHIR build failure @ Wed, Sep 12, 2018 23:24+0000                          0.106  39sec 1162MB
2018-09-12T23:24:18.2115632Z      [java] Error: Error Parsing Resource patient: unable to read /tmp/ohfu-fhir4115606082990555608xml: /tmp/ohfu-fhir4115606082990555608xml (No such file or directory)
2018-09-12T23:24:18.2137582Z      [java] java.lang.Exception: Error Parsing Resource patient: unable to read /tmp/ohfu-fhir4115606082990555608xml: /tmp/ohfu-fhir4115606082990555608xml (No such file or directory)
2018-09-12T23:24:18.2165371Z      [java]    at org.hl7.fhir.definitions.parsers.SourceParser.loadResource(SourceParser.java:1112)
2018-09-12T23:24:18.2188562Z      [java]    at org.hl7.fhir.definitions.parsers.SourceParser.parse(SourceParser.java:258)
2018-09-12T23:24:18.2204235Z      [java]    at org.hl7.fhir.tools.publisher.Publisher.execute(Publisher.java:591)
2018-09-12T23:24:18.2219542Z      [java]    at org.hl7.fhir.tools.publisher.Publisher.main(Publisher.java:482)
2018-09-12T23:24:18.2238740Z      [java] Caused by: java.lang.Exception: unable to read /tmp/ohfu-fhir4115606082990555608xml: /tmp/ohfu-fhir4115606082990555608xml (No such file or directory)
2018-09-12T23:24:18.2257560Z      [java]    at org.hl7.fhir.definitions.model.Example.<init>(Example.java:118)
2018-09-12T23:24:18.2273987Z      [java]    at org.hl7.fhir.definitions.parsers.SpreadsheetParser.readExamples(SpreadsheetParser.java:1596)
2018-09-12T23:24:18.2291583Z      [java]    at org.hl7.fhir.definitions.parsers.SpreadsheetParser.parseResource(SpreadsheetParser.java:488)
2018-09-12T23:24:18.2308114Z      [java]    at org.hl7.fhir.definitions.parsers.SourceParser.loadResource(SourceParser.java:1110)
2018-09-12T23:24:18.2325972Z      [java]    ... 3 more
2018-09-12T23:24:18.2354723Z      [java] Caused by: java.io.FileNotFoundException: /tmp/ohfu-fhir4115606082990555608xml (No such file or directory)
2018-09-12T23:24:18.2375827Z      [java]    at java.io.FileInputStream.open0(Native Method)
2018-09-12T23:24:18.2391406Z      [java]    at java.io.FileInputStream.open(FileInputStream.java:195)
2018-09-12T23:24:18.2406651Z      [java]    at java.io.FileInputStream.<init>(FileInputStream.java:138)
2018-09-12T23:24:18.2422650Z      [java]    at org.hl7.fhir.utilities.CSFileInputStream.<init>(CSFileInputStream.java:54)
2018-09-12T23:24:18.2441195Z      [java]    at org.hl7.fhir.definitions.model.Example.<init>(Example.java:115)
2018-09-12T23:24:18.2456675Z      [java]    ... 6 more
2018-09-12T23:24:18.2464314Z

Grahame Grieve (Sep 12 2018 at 23:27):

oh interesting

Grahame Grieve (Sep 12 2018 at 23:27):

probably they are.

Grahame Grieve (Sep 12 2018 at 23:27):

I'll.. investigate. Could be painful

Josh Mandel (Sep 13 2018 at 14:15):

OK -- if it is, le tme know; I can just isolate in containers, probably.

Ewout Kramer (Sep 17 2018 at 16:04):

"What I haven't figured out with Git is: how do I do this? It seems that my clone is tied to a single branch, and I can't commit to another branch locally without some... operation.. of some sort that might or might not trash what I'm doing.

When @Ewout Kramer sold me on it, it was because I'd get more control over how to commit my work, but right now it seems that I have a lot less"

Yes, I was talking about staging - which I have not seen in the discussion so far. I am sometimes working on multiple fixes at the same time. When part of it works, I stage only those files that are concerned with that fix and then commit them. Much like the SVN tools allowed you to pick a subset of the changed files to commit. So like this, you could be working on multiple issues - periodically run the build, and only commit those changes that you are actually happy with.

Ewout Kramer (Sep 17 2018 at 16:07):

It seems that my clone is tied to a single branch

Yes, though you could check out multiple copies of the repo. It's not normally something I do in my workflows though - I switch branches since it's pretty lightweight.

and I can't commit to another branch locally without some... operation

You cannot commit "blindly" to another branch - there might be merge conflicts (at the least) when you'd do that. The closest thing is to copy the content of once commit of another branch onto your current branch (cherry-picking).

Lloyd McKenzie (Sep 17 2018 at 16:12):

Switching branches isn't lightweight if you can't do so when you're running a 45-minute build - which is why (for at least some individuals) we're going to move to a single commit branch where all changes (over a period of weeks or months) are pushed to a user-specific branch rather than issue-specific branching.

Grahame Grieve (Sep 17 2018 at 22:30):

, I was talking about staging - which I have not seen in the discussion so far

@Ewout Kramer please explain

Rob Hausam (Sep 18 2018 at 15:55):

I expect that Ewout is talking about staging for a commit - via git add (he can confirm - or not). We haven't mentioned that very much, but it is the way to specify which of your changes are included in a specific commit.

Claude Nanjo (Sep 19 2018 at 18:27):

Please update the documentation on the github wiki regarding the commit process as it is now outdated. It should provide guidance on the pull request approach and associated branching requirements.

Bryn Rhodes (Sep 19 2018 at 19:19):

Hi @Claude Nanjo ! There's a git-help stream specifically for github related questions

Bryn Rhodes (Sep 19 2018 at 19:20):

There's a Getting Started page on the fhir wiki for using git in general, and there's a TortoiseGit page that talks about using TortoiseGit on Windows specifically.

Claude Nanjo (Sep 19 2018 at 20:21):

I think the Getting Started guide may need an update.

Claude Nanjo (Sep 19 2018 at 20:22):

When building and then getting ready to stage my changes, I get two new vscache files: all-systems.cache and lang.cache. I assume these should be added to gitignore? Also, I get a whole bunch of .png files. I am assuming these too should be in gitignore?

Claude Nanjo (Sep 19 2018 at 20:29):

Correction: the png files are new but the vscache files are modified. I am adding a *.png for now to gitignore since it is a pull request but someone should probably update it and refine it unless these files generated files should be committed.

Grahame Grieve (Sep 19 2018 at 23:40):

it's on my list to get to this. I'm impossibly far behind right now. but in general the cache files should in in git, and the other file should not exist

Ewout Kramer (Sep 25 2018 at 09:04):

, I was talking about staging - which I have not seen in the discussion so far

@Ewout Kramer please explain

I'll take the opportunity that we will be face to face in a few days to show you and answer any questions that might remain ;-)

Josh Mandel (Oct 08 2018 at 15:02):

I am planning to move our git instructions into the hl7 confluence. I don't think there's an existing space for it and I don't think I have permissions to create one. @Lloyd McKenzie can you please help?

Josh Mandel (Oct 08 2018 at 15:09):

Also, I tried creating a page just to test out formatting etc, and I can't even delete or move it (I lack permissions).

Josh Mandel (Oct 08 2018 at 15:10):

So if you can help me delete this page that'd be great.

Lloyd McKenzie (Oct 08 2018 at 15:25):

Before we start moving stuff to Confluence, I'd like both the FMG and FHIR-I to spend some time thinking about exactly what spaces we want, what should go in them, what we should bring across, etc. I was thinking we'd start having those discussions after the freeze deadline for content submissions. Do we think there's a reason to move the Git instructions sooner than that?

Josh Mandel (Oct 08 2018 at 16:35):

No rush on my side; I was just trying to be responsive to the request from last week. (FYI @Grahame Grieve.)

Josh Mandel (Oct 08 2018 at 16:36):

If you can delete the page I linked above though, that'd be great to avoid spreading info into too many places before we're ready to make the cutover.

Lloyd McKenzie (Oct 08 2018 at 16:39):

Deleted

Last updated: Apr 12 2022 at 19:14 UTC