git boot camp syncing with upstream instructions need to rebase the branch for proper PR review

[gpshead]

The instructions at https://cpython-devguide.readthedocs.io/gitbootcamp.html#syncing-with-upstream tell you how to pull changes into your forked repository and how to pull them into your branch that you may have made a PR out of. But it doesn't deal with rebasing your branch so that the code review on your branch does not contain extraneous changes when you have re-synced with the upstream repo.

This seems necessary in many PRs that go on for a while where you need to sync and merge across changes already made upstream (or potentially fix additional issues since introduced upstream in your PR).

I successfully used these instructions to rebase my branch used in my PR to get its diffs/changes back into a sane reviewable state again after I had already followed and pushed via the existing syncing with upstream instructions in my-branch:
https://www.digitalocean.com/community/tutorials/how-to-rebase-and-update-a-pull-request

Looking at my bash history, this involved:

1. git merge-base my-branch master to find the commit hash i wanted to rebase the branch off of.
2. git checkout my-branch
3. git diff master >backup.diff` as I was entering new territory for me (relatively inexperienced git user).
4. 'git rebase -i $hash_from_the_merge-branch_command_above` followed by not understanding why there were duplicate commits in my list. i suspect should've removed the later copies of those during this process; oops.
5. many iterations of git rebase --skip and git reset to skip past the duplicates that would otherwise show up as useless empty commits until the rebase was done.
6. some final git diff commands to confirm that it appeared to have worked.
7. git push origin my-branch

This should be able to be simplified. All of the rebase skipping and reset stuff was effectively manual recovery work from whatever state I had wound up in. I hope that is not normal if a rebase is done before making follow on changes after syncing with upstream. (?)

There may be other ways to do this. This is what I've happened to figure out so far.

[willingc]

@gpshead Here's the workflow that I typically use with git and GitHub (It's the second part of your referenced link - though I'll add a bit more commentary here). I'm assuming that your fork is named origin and the cpython repo is upstream.

1. Let's say I submitted a PR a month ago that I developed on my pr-april branch.

2. There's been a month worth of changes to the master branch which I had submitted the PR against.

3. If I want to move the PR on the pr-april branch to the HEAD of master, I would enter:

   git checkout pr-april
   git fetch upstream
   git rebase upstream/master
   

   At this point I would correct any merge conflicts, git add the changed files. After correcting
   merge conflicts, I will git rebase --continue. You may need to do this several times if there are
   multiple merge conflicts.

4. At this point, I would do a git log to check the history. If all looks good, I would:

   git push -f origin pr-april
   

   [Note: This assumes that others are not working on your branch.]

And, yes, there are multiple ways to accomplish the same result.

[gpshead]

I think I'll have to play around with my branches during future PRs to have any hope of understand what happened.

After your step 3 git rebase upstream/master I had some error during the git push origin during which I believe git suggested I needed to pull from upstream first. So I did a git pull followed by another git push origin pr-april (suspiciously no --force was needed?). After this I continued to make my own changes in the branch and eventually pushed. At which point my PR contained my changes as well as several others that came in during the rebase upstream/master.

Was my git pull in the middle the mistake there? Did that perhaps undo my rebase and leave my branch marked as coming from its original location already stored upstream but with the merged upstream commits on top of it?

[willingc]

Was my git pull in the middle the mistake there?

Probably, it did a merge. Personally, I rarely git pull unless it is set either globally to rebase or in the command to git pull --rebase. The first few paragraphs here may help: https://git-scm.com/docs/git-pull

FYI...Here's my opinionated workflow when I teach git to new devs. I've found that for those starting out there's less merge conflicts by always developing on a branch and using fetch/rebase.

[Mariatta]

After your step 3 git rebase upstream/master I had some error during the git push origin

I think you need to do git push origin --force here to avoid the error.
I just made a PR in the devguide to update this section https://github.com/python/devguide/pull/214/files

[willingc]

FYI... --force only needs to be used if you are modifying the branch history. If you are rebasing your branch to the HEAD of upstream/master, then frequently the --force option is not necessary.

[csabella]

@willingc Thanks for that presentation. I wish I could also hear you talking about each slide. I'm sure there are nuggets that aren't written down. With your 'real-world analogy', would 'git push' be like the plane arriving at the destination? And then creating the PR would be like going through customs?

I'm slowly understanding some of git, but I know what @gpshead is experiencing. This project was the first time I've used git and even with Stack Overflow, I really broke it a few times. Probably the 'git pull' was the main problem from what Carol said. But, I had messages that I had no idea how to resolve. More than once, I deleted my fork and started over. :-(

I currently have some PRs that are over 2 months old and I'm afraid to rebase them. But, as I said, I'm slowly learning git and getting a hang of syncing with upstream is my current battle.

One time I synced a branch, then I had to make another change and commit. When I commited and pushed the change, my github repo showed all the commits that everyone had done. There were like 60 commits. lol Somehow I googled the right SO to get me out of that, but I went into full on panic mode thinking I had broken my PR.

[brettcannon]

@csabella I have had to delete my clones numerous times on every VCS I have ever used.

If you're worried about losing work you can always save a patch first and then do the rebase.

[csabella]

@brettcannon Thanks! Knowing that makes me feel a whole lot better! I felt like I had failed because I couldn't figure out any other way to fix it.

[willingc]

@csabella Thanks for the kind words. I have one more week of travel but after that I would be happy to pair with you to go through the git talk or just work through a few rebases.

One older resource that I used a lot the first year that I was working with git is http://sethrobertson.github.io/GitFixUm/fixup.html I share it since it shows the many ways that I, much like @brettcannon, had to troubleshoot the huge mess from pulls and merges.

Many times, it is simpler (and less time consuming) to reclone in a new directory.

[csabella]

@willingc Thank you. I actually have 5 PRs open right now, so yesterday I worked through your steps for updating my branches and all of them worked with no issues! Without the git push -f origin pr-april step, I didn't understand why it didn't work. Now I have a visual of what rebase is doing and I'm not afraid of losing my changes after a rebase and a push.

I will definitely bookmark that other link you sent. The first line alone is gold - "recovering from what you did not mean to do when using git". LOL That's exactly how I've felt using it. When it gives an error, it's like, what did I do now????

Tutorials are great, but most of them are a step by step without introducing any real life situations of what happens when git tells you no, it won't do what you ask until you fix it. I'm beginning to understand that it was a language and terminology barrier. The git docs might be great at explaining what to do if this or if this or if that, but if you don't know what scenario you're in, it's impossible to follow.

Python is so much easier. ;-)

[AA-Turner]

The referenced section no longer exists on https://devguide.python.org/getting-started/git-boot-camp/, as far as I can tell, I assume the page has been updated. Closing.