Github Fork-n-Beans Workflow

Owned by Chris Kirkwood-Watts
Last updated: Sept 28, 2018 by Medha Parlikar (Unlicensed)

Audience

This document is intended for full-time developers, community contributors, and projects managers who all need to understand the details of how to use Github effectively in the RChain Cooperative's environment. Full-time developers and other contributors must follow these guidelines to the extent possible.

Summary rules of etiquette

  • All work is code reviewed, so please familiarize yourself with that process thoroughly.
  • Ensure that you are keeping your personal branches up to date with the main dev branch so that your reviewer will only have to see your changes. See below more details.
  • For each release, there must be a branch "release/x.y.z", all release branches must be merged back to "dev" branch.
  • Your PR will be the basis for kicking off a code review.  
  • In your PR, include a link to your Jira ticket in the comments.
  • When creating your PR use smart commits, and include the Jira issue number.
  • Sign your commits (see How to sign commits to rchain/rchain)
  • Select a reviewer.
  • Validate that unit and integration tests passed on your PR.   
  • Wait for your reviewer's feedback.
  • Address any feedback.
  • Once approved, merge to dev.

If you're picking up someone else's pull request, then whatever you do must preserve the commit history. If you can accomplish that by rebasing off someone else's PR, then great. If you need to merge the PR and then create a new PR, then that's also fine. Merging a broken PR and then fixing it "later" is the least preferable, but there are situations where this also works. E.g., if the "fix" is already implemented and will be merged almost immediately.


Please do NOT:

  • Please do not close PRs when there are new changes which result in conflicts. You can add changes and resolve conflicts by merging or rebasing.

The basics of forking

The RChain Cooperative has a variety of repositories under https://github.com/rchain and at various times you may be required to contribute to each of them. This is a simple git workflow that keeps all your work on branches in your own fork of the repository. In the general case, no one ever pushes to rchain/rchain, which keeps our shared repository clean. All git workflows require diligence to maintain good hygiene around branching, issuing pull requests, and merging. The main rules to keep in mind for this one are:

  1. Never ever mess with master on rchain/rchain.
  2. In fact, only ever push to your own fork!

One great piece of advice is to make your personal fork be origin and add the upstream (rchain/rchain) fork as a remote. That helps you avoid accidentally impacting the wrong fork by default. Either way you do it, the network diagram will look something like this:


Example process

Setup

Make a personal fork of rchain/rchain on github; my user is kirkwood, so I forked to kirkwood/rchain. On your development machine, clone your personal branch (kirkwood/rchain for me) and add rchain/rchain as a remote called upstream.

Clone & Add Remote
<dev:~/src> git clone git@github.com:kirkwood/rchain
Cloning into 'rchain'...
remote: Counting objects: 2100, done.        
remote: Compressing objects: 100% (24/24), done.        
remote: Total 2100 (delta 10), reused 34 (delta 7), pack-reused 2058        
Receiving objects: 100% (2100/2100), 1.52 MiB | 10.25 MiB/s, done.
Resolving deltas: 100% (791/791), done.
<dev:~/src> cd rchain
<dev:~/src/rchain (master)> git remote add upstream git@github.com:rchain/rchain
<dev:~/src/rchain (master)> git fetch --all
Fetching origin
Fetching upstream
remote: Counting objects: 316, done.        
remote: Compressing objects: 100% (89/89), done.        
remote: Total 316 (delta 89), reused 198 (delta 69), pack-reused 87        
Receiving objects: 100% (316/316), 71.69 KiB | 1.79 MiB/s, done.
Resolving deltas: 100% (100/100), completed with 23 local objects.
From github.com:rchain/rchain
 * [new branch]      dev                      -> upstream/dev
 * [new branch]      dev-kent-arithmetic-test -> upstream/dev-kent-arithmetic-test
 * [new branch]      dev-kent-move            -> upstream/dev-kent-move
 * [new branch]      master                   -> upstream/master
<dev:~/src/rchain (master)> git remote -v
origin	git@github.com:kirkwood/rchain (fetch)
origin	git@github.com:kirkwood/rchain (push)
upstream	git@github.com:rchain/rchain (fetch)
upstream	git@github.com:rchain/rchain (push)

Feature branch

Create a new branch, tracking upstream/dev (in other words, branch off of dev in the main rchain repository). Here I'm working on Jira issue CORE-23:

Create Feature Branch
<dev:~/src/rchain (master)> git checkout -b CORE-23 upstream/dev --track
Branch CORE-23 set up to track remote branch dev from upstream.
Switched to a new branch 'CORE-23'

Do some work. Ensure the feature branch is up-to-date with the main dev branch, and commit your changes.

Commit Locally
<dev:~/src/rchain (CORE-23)> git pull
Already up-to-date.
<dev:~/src/rchain (CORE-23)> git status
On branch CORE-23
Your branch is up-to-date with 'upstream/dev'.

Untracked files:
  (use "git add <file>..." to include in what will be committed)

	comm/src/main/scala/coop/rchain/comm/foo.scala

nothing added to commit but untracked files present (use "git add" to track)
<dev:~/src/rchain (CORE-23)> git add comm/src/main/scala/coop/rchain/comm/foo.scala 
<dev:~/src/rchain (CORE-23)> git commit -m'Added a foo.'
[CORE-23 2828de5] Added a foo.
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 comm/src/main/scala/coop/rchain/comm/foo.scala

Really, make sure your feature branch is up-to-date with dev. Then, push the feature branch to your personal fork.

Push Feature Branch
<dev:~/src/rchain (CORE-23)> git pull
Already up-to-date.
<dev:~/src/rchain (CORE-23)> git remote -v
origin	git@github.com:kirkwood/rchain (fetch)
origin	git@github.com:kirkwood/rchain (push)
upstream	git@github.com:rchain/rchain (fetch)
upstream	git@github.com:rchain/rchain (push)
<dev:~/src/rchain (CORE-23)> git push origin CORE-23
Counting objects: 270, done.
Delta compression using up to 12 threads.
Compressing objects: 100% (95/95), done.
Writing objects: 100% (270/270), 62.08 KiB | 31.04 MiB/s, done.
Total 270 (delta 75), reused 246 (delta 63)
remote: Resolving deltas: 100% (75/75), completed with 10 local objects.        
To github.com:kirkwood/rchain
 * [new branch]      CORE-23 -> CORE-23

Sign your commits

See How to sign commits to rchain/rchain for information on how to sign your commits.

Pull request

Issue a pull request on Github from your newly pushed feature branch to dev. Not to master, to dev. Github may try to trick you.


Keeping up-to-date and resolving merge conflicts

Sometimes, once you've created a pull request, things will happen in the upstream repository before you get a chance to merge. This can cause your local code and your fork to become out of date with the upstream repository. Some people are tempted to do a rebase at this point. Do not give in to the dark side of the git. You are going to have to fix the merge conflicts one way or the other. You want to fix them as the result of a merge command, not as the result of a rebase.

Begin by bringing your local copy up-to-date with upstream.

/usr/local/pyrofex/leaf/rchain
leaf@lisbeth $ git pull --all
Fetching origin
Fetching upstream
remote: Counting objects: 496, done.
remote: Compressing objects: 100% (98/98), done.
remote: Total 496 (delta 215), reused 290 (delta 184), pack-reused 158
Receiving objects: 100% (496/496), 82.91 KiB | 0 bytes/s, done.
Resolving deltas: 100% (249/249), completed with 47 local objects.
From github.com:rchain/rchain
   c147603..e73cc9a  JosDenmark-patch-1     -> upstream/JosDenmark-patch-1
 * [new branch]      birch-tmp-tut-typo-fix -> upstream/birch-tmp-tut-typo-fix
   8300ad9..f7a0d81  dev                    -> upstream/dev
   154bd8d..e0178ca  dev-eitan&kyle-spatial -> upstream/dev-eitan&kyle-spatial
 * [new branch]      dev-eitan-structural   -> upstream/dev-eitan-structural
   d2b59e3..e015da4  master                 -> upstream/master
 * [new tag]         rholang-0.1            -> rholang-0.1
Fetching kent
remote: Counting objects: 1, done.
remote: Total 1 (delta 0), reused 0 (delta 0), pack-reused 1
Unpacking objects: 100% (1/1), done.
From github.com:KentShikama/rchain
 * [new branch]      dev-kent-add-new-line -> kent/dev-kent-add-new-line
Already up-to-date.

In this case, my local copy was already up to date with my fork, but there was activity on upstream branches and that activity will need to be merged. I'm current on the upstream/dev branch, so I will merge that.

leaf@lisbeth $ git merge upstream/dev                                                            
Updating ac89d89..f7a0d81                                                                          
Fast-forward                                                                          
 docker/rholang-cli/run-rhoscala                                 |   12 +-       
 docker/rholang-web/Dockerfile                                   |    4 +-            
 docker/rholang-web/run-server                                   |   12 +              
... many more lines elided ...
 create mode 100644 storage/src/test/scala/coop/rchain/storage/regex/PathRegexUnitTests.scala
 delete mode 100644 storage/src/test/scala/coop/rchain/storage/stores/storeFlat.txt
 delete mode 100644 storage/src/test/scala/coop/rchain/storage/stores/storeNested.txt
 delete mode 100644 storage/src/test/scala/coop/rchain/storage/stores/storeRecursive.txt
 create mode 100755 storage/test.sh

If you get a merge conflict, you should resolve it in the usual way. If you need help resolving a merge, please ask in the discord before you push. You may wish to check your status now, to see how all the commits that your Github fork doesn't know about.

/usr/local/pyrofex/leaf/rchain
leaf@lisbeth $ git status
On branch dev
Your branch is ahead of 'origin/dev' by 108 commits.
  (use "git push" to publish your local commits)
nothing to commit, working tree clean

At this point, you local copy is fully committed, you have merged upstread, and the only thing left to do is to bring the Github fork up-to-date with everything else. You do that by pushing to it in the usual way.

/usr/local/pyrofex/leaf/rchain
leaf@lisbeth $ git push
Counting objects: 855, done.
Delta compression using up to 12 threads.
Compressing objects: 100% (300/300), done.
Writing objects: 100% (855/855), 136.38 KiB | 0 bytes/s, done.
Total 855 (delta 395), reused 801 (delta 343)
remote: Resolving deltas: 100% (395/395), completed with 56 local objects.
To github.com:nashef/rchain.git
   ac89d89..f7a0d81  dev -> dev

Nota Bene: This push will automatically update any pull requests from your fork→rchain/rchain. In my case, if I have a pull request open from nashef/rchain→rchain/rchain, then the pull request will now contain the differences between the upstream and forked branches.

This process is usually simpler and more reliable than a rebase. You can try a rebase, but it is tricky to get right. If you do it incorrectly, you can change the attribution of large amounts of code, which is deeply undesirable. I know many developers use rebase all the time, but in all my years of experience with git, I haven't had to do it more than a handful of times.


Forking as it relates to releases

The basic approach to forking extends to how we prepare for and release software. Below is a high-level view of the process taken from https://nvie.com/posts/a-successful-git-branching-model. The SRE team supports the merge from dev to release branches and master, and hotfixes as needed.

  • Release branch is cut from Dev at feature complete for the release *(Note, features that are not ready, will be de-scoped from release)
    • Test release branch & bug fix if needed
    • Bug fixes will require 2 Pull Requests
      • 1 to the release branch
      • 1 to dev branch
  • Push release to Master
  • Tag release in Master
  • Release from Master