Stream: committers/git-help
Topic: conflicting files error
Craig Newman (Nov 01 2018 at 22:52):
My pull request as conflict (in files I didn't (intend to) change) - https://github.com/HL7/fhir/pull/212
Conflicting files
tests/ig30/txCache/all-systems.cache
vscache/all-systems.cache
vscache/lang.cache
vscache/ucum.cache
I know there's been a large vscache thread today but don't know if this is the same issue or not.
Any ideas how I fix this? Thanks
Grahame Grieve (Nov 01 2018 at 23:02):
just revert them
Craig Newman (Nov 01 2018 at 23:06):
Not sure how to do that.
Craig Newman (Nov 01 2018 at 23:06):
I'm using TortoiseGit if that makes a difference
Lloyd McKenzie (Nov 01 2018 at 23:40):
I don't know how to revert once you get to the point of seeing a conflict in a pull request - I always select "resolve conflicts" and then edit each file - but that requires a situation where it's obvious what edits to make - and for the cache files, I don't know if that's the case...
Craig Newman (Nov 01 2018 at 23:50):
The changes were minimal so I just made a new clone and re-established the changes in ImmunizationRecommmendation. I'm doing the local build now and once that's done, hopefully can just do a new commit and pull request. Do you think that will work? how do I clean up the existing pull request? Do I just click the "close" button?
Eric Haas (Nov 02 2018 at 01:51):
see my post here https://chat.fhir.org/#narrow/stream/138-committers.2Fgit-help/subject/Getting.20rid.20of.20.20vscache.20conflict
Grahame Grieve (Nov 02 2018 at 23:13):
now I'm stuck. I've created a new branch, from master, fully synced, rebuilt the terminology cache, and committed it. here's the PR: https://github.com/HL7/fhir/pull/218
Grahame Grieve (Nov 02 2018 at 23:14):
this must overwrite the terminology cache... but git, for some reason I cannot imagine, says that this is a merge conflict.
Grahame Grieve (Nov 02 2018 at 23:14):
how do I force git to just use my new versions?
Grahame Grieve (Nov 02 2018 at 23:15):
@Josh Mandel Igit remains a big obstacle to getting work done. I have had some impassioned approaches from the other committers about this.
Rob Hausam (Nov 02 2018 at 23:15):
I just pulled your changes and am building it locally now.
Grahame Grieve (Nov 02 2018 at 23:15):
and that will help?
Grahame Grieve (Nov 02 2018 at 23:21):
I see that the PR history merges with other branches. I have no idea why
Rob Hausam (Nov 02 2018 at 23:21):
Well, my build won't. But I stashed my local files so I could avoid the local conflict and pull. If I'm understanding where you are in the process I think you can probably do a force merge?
Grahame Grieve (Nov 02 2018 at 23:22):
how do I do that?
Rob Hausam (Nov 02 2018 at 23:23):
not sure of the steps with TortoiseGit (if you're using that)
and I always double check the syntax before doing something like this
Grahame Grieve (Nov 02 2018 at 23:23):
I don't have merge conflicts locally. They're on github....
Rob Hausam (Nov 02 2018 at 23:24):
yes, I gathered that was your issue
Rob Hausam (Nov 02 2018 at 23:24):
I need to run for a moment and I'll be back in a few minutes
Josh Mandel (Nov 03 2018 at 00:59):
Re: forcing git to use new versions, do you mean on GitHub? You can locally create a new branch off master and updated these files in your new branch, then push and merge into master. You'll see conflicts if the files change in master before you merge.
Josh Mandel (Nov 03 2018 at 01:00):
Re: the broader issues, I think the key question is whether we're getting better, as a community.
Josh Mandel (Nov 03 2018 at 01:01):
https://github.com/HL7/fhir/pulse tells a somewhat encouraging story for this week... But it's obviously tough.
Grahame Grieve (Nov 03 2018 at 01:16):
We’re not getting happier. As for my PR - I’ve already tried twice, for 6 hours of work. I know what the changes are, and prepared the commit... would have been enough in svn days. That’s the common complaint from committers: things that used to be simple and take a few minutes now take hours of work
Grahame Grieve (Nov 03 2018 at 01:17):
And tax our ability to deal with complexity. Just another thing we have to deal with
Rob Hausam (Nov 03 2018 at 01:20):
My comment about "force merge" wasn't really accurate - that doesn't actually exist, but other similar things do. I think the bottom line is that if GitHub won't allow you to merge (for whatever it's reasons are) then you need to do that locally and then push back up. You can use Josh's suggestion or other options locally.
Lloyd McKenzie (Nov 03 2018 at 01:21):
It's weird that when a conflict is detected you can't, from the Git website just say "mine wins"
Lloyd McKenzie (Nov 03 2018 at 01:21):
Obviously you'd need to be cautious about using that sort of thing, but it's sometimes necessary
Josh Mandel (Nov 03 2018 at 01:21):
The website doesn't have rich tools for managing conflicts. 6 months ago it did not have any tools for managing conflict. This is a workflow where typically you would handle the merge locally and push once it was handled.
Rob Hausam (Nov 03 2018 at 01:22):
Yes, that would be nice to have - and I've seen others complaining about the same thing.
Josh Mandel (Nov 03 2018 at 01:23):
In any event, if we don't feel like things are improving and people don't like the experience, it seems like we should go back to svn.
Josh Mandel (Nov 03 2018 at 01:23):
I personally think it would be a disappointing outcome, but hey we tried the experiment.
Josh Mandel (Nov 03 2018 at 01:24):
I really liked seeing changes submitted in branches, reviewable and mergeable on their own time, but the usability of the overall system may be too low.
Rob Hausam (Nov 03 2018 at 01:26):
I think the build has been more stable and reliable since we made the switch - and that was one of the main goals.
Josh Mandel (Nov 03 2018 at 01:27):
Yes, master has been quite stable. Perhaps even zero broken builds?
Rob Hausam (Nov 03 2018 at 01:27):
Yes, I think that's right. It's always building successfully for me now.
Lloyd McKenzie (Nov 03 2018 at 02:22):
The notion of managing conflicts locally doesn't work well when the gap between creating a PR and being able to merge it is 30-45 minutes (and that's only if you merge as soon as the PR passes). Also, if I've been working on a change locally for several days (or even over a week), it's not obvious how I merge my changes with the remote master before pushing without switching out of my branch. Perhaps that should be part of the standard workflow before pushing a change?
Josh Mandel (Nov 03 2018 at 02:24):
Re: managing changes locally, you can run tests as part of the normal pull request process and then handle the merge itself locally. I'd be happy to work through this with you and then we could write up instructions.
Josh Mandel (Nov 03 2018 at 02:24):
As for the longer term changes, yes a good habit is to keep merging Master into your branch so that he you don't run into conflicts in the first place.
Rob Hausam (Nov 03 2018 at 02:26):
Actually, what Josh just said is what I was going to say. You can do a fresh pull of the master branch and then merge (or possibly rebase) the latest master into your feature branch before you push and generate the PR. That should avoid the conflicts.
Lloyd McKenzie (Nov 03 2018 at 02:46):
How do I do a pull of the master and merge it into my branch without switching out of my branch?
Lloyd McKenzie (Nov 03 2018 at 02:47):
(Preferably using TortoiseGit)
Josh Mandel (Nov 03 2018 at 02:48):
"git fetch" and "git merge origin/master" from the cli... I'm sure there is a nice way to do this in Tortoise
Grahame Grieve (Nov 03 2018 at 06:06):
So I’m away for the weekend and various restrictions to power access and time mean I can’t finish this pr.. can someone run it to the ground? The changes that matter are the java and XML changes. The cache files... anyone can just delete them before the build, and then commit what they end up with.
Grahame Grieve (Nov 03 2018 at 06:08):
I don’t really want to go back. The inefficiencies are at least partly because you can’t try to do things the same way with git and it takes time to learn how to navigate the choices. But git is very obnoxious about conflicts in some parts of the workflow... clearly it’s not designed for people who actually use it.
Josh Mandel (Nov 03 2018 at 12:12):
I'll run it to the ground, yes:-)
Josh Mandel (Nov 03 2018 at 12:37):
Er, I can't do this without help -- the cache currently in master looks totally broken, like, https://github.com/HL7/fhir/blob/master/vscache/all-systems.cache starts with:
-------------------------------------------------------------------------------------
// <30 newlines snipped>
{"hierarchical" : true, "valueSet" :{
"resourceType" : "ValueSet",
"compose" : {
"include" : [{
"system" : "http://snomed.info/sct"
},
{
"system" : "http://loinc.org"
},
{
"system" : "http://www.nlm.nih.gov/research/umls/rxnorm"
}]
}
}}
####
e: {
"valueSet" : {
Or maybe that's normal?? Are these not json files?
Josh Mandel (Nov 03 2018 at 12:38):
Yeah, normal I guess. The build errors are pretty cryptic :)
Josh Mandel (Nov 03 2018 at 12:38):
I also see windows-style line endings on these cache files; is that intentional ?
Josh Mandel (Nov 03 2018 at 16:07):
Anyway, made these changes and merged 'em (https://github.com/HL7/fhir/pull/220)
Josh Mandel (Nov 03 2018 at 17:20):
We seem to still be thrashing on these cache files though, @Grahame Grieve. e.g., @Paul Knapp has pull requests that are generating conflicts on these cache files. I don't understand how this is supposed to work.
My understanding was that pretty much only Grahame should eve be pushing changes to cache files; is that right? If so, we should make sure these files aren't generally modified when other people build; if not we should come up with some other strategy for how we're going to avoid conflicts and thrashing on these files.
Lloyd McKenzie (Nov 03 2018 at 18:14):
What used to happen with SVN is that conflicts with the cache files would (typically) just merge and we wouldn't have conflicts. The question is why that's not happening under Git. Essentially everyone who makes changes to their models has the potential to trigger different terminology needs and thus change cache contents. When they commit, their changes should merge with whatever's in master. But that seems to not be happening cleanly.
Josh Mandel (Nov 03 2018 at 18:16):
I think in subversion there was never a need to merge... because people were always building on the latest? Or are you saying svn knew how to do merges on this file type (which is... Kind of ad hoc) beyond simple line oriented merge logic?
Josh Mandel (Nov 03 2018 at 18:17):
Either way, you're suggesting that everyone reallocate does need to be editing these cache files routinely in their own commits.
Josh Mandel (Nov 03 2018 at 18:17):
Which wasn't what I had understood (just my error, most likely)
Josh Mandel (Nov 03 2018 at 18:22):
If we want merges to work better on these files, we could define a custom merge driver for whatever the cache file format is (some kind of multipart json thingy).
Lloyd McKenzie (Nov 03 2018 at 18:26):
Nobody edits them - they're generated based on data looked up from the terminology server. Caching the information speeds the builds significantly, so you want the information cached from each individual's build to be merged into master. The challenge with having the committers try to manually merge is no committer has a clue what's in the file - they're created by the build process
Lloyd McKenzie (Nov 03 2018 at 18:27):
I don't know why we seemed to have less collision issues in SVN
Josh Mandel (Nov 03 2018 at 18:28):
I'm just saying that if they're automatically updated in contradictory ways, we could define a custom merge driver that would help git do the right thing (more often).
Lloyd McKenzie (Nov 03 2018 at 18:31):
If that's an option, that certainly sounds like something that would be useful
Josh Mandel (Nov 03 2018 at 18:33):
It's possible. The major downside is that it would mean supplying a merge program that everyone would have to install and have on their path. Simpler might be to define a cache file format that is more like oriented, so it merges more cleanly by default.
Lloyd McKenzie (Nov 03 2018 at 18:58):
Seeing as we generally hit the merge conflict on the Github website, can't we just run it from the online Git?
Josh Mandel (Nov 03 2018 at 19:00):
GitHub doesn't currently support running arbitrary binaries in their automated merge process.
Grahame Grieve (Nov 03 2018 at 19:23):
I don’t understand this - it’s just a text format, and should merge like text. And there shouldn’t be new merge conflicts unless people are not fully synchronised
Grahame Grieve (Nov 03 2018 at 19:24):
And yes , windows style line endings
Grahame Grieve (Nov 03 2018 at 19:26):
Lloyd is right. The only time these files change should be when you change something about the terminology is used, and you should commit the result to save other people to time to make the call
Josh Mandel (Nov 03 2018 at 19:41):
The "text format" includes stuff like json, which doesn't necessarily merge great when treated as pure text.
Josh Mandel (Nov 03 2018 at 19:42):
For line endings: do we _want_ windows style line endings on the files? Is that, erm, part of their file format specification?
Paul Knapp (Nov 03 2018 at 20:08):
The snomed, ucum, loinc files I am not referring to, they simply get changed if I build.
Grahame Grieve (Nov 04 2018 at 04:02):
Yes but don’t commit them for now. They should stop changing.
Grahame Grieve (Nov 04 2018 at 04:03):
The txt format won’t ever need to merge inside the json. They merge just fine when they build correctly. I wish java had a mode where depending on the default encoding is an error
Grahame Grieve (Nov 04 2018 at 04:04):
I have to choose one line ending style so we don’t have line ending merge clashes which would be beyond stupid. So I chose windows style, since some windows text utilities don’t handle Unix style line endings
Joel Schneider (Nov 05 2018 at 17:38):
It may be worth mentioning that Git also has facilities for dealing with line endings, which could potentially interfere or interact with other strategies for dealing with them.
https://help.github.com/articles/dealing-with-line-endings/#platform-all
Grahame Grieve (Nov 05 2018 at 20:29):
We should never let git change line endings on us
Lloyd McKenzie (Nov 05 2018 at 20:43):
I know Git has informed me that it was changing line endings when I stashed stuff. How do I tell it to not mess with line endings?
Joel Schneider (Nov 05 2018 at 21:40):
Using a .gitattributes file to configure file types as binary is one option. However, that could also affect how git diff works, so additional tweaking may be needed.
Grahame Grieve (Nov 06 2018 at 06:36):
I get asked when installing git
Last updated: Apr 12 2022 at 19:14 UTC
