Skip to content

Basing a commercial product on FreeBSD using git.

November 24, 2012

With the advent of DVCS and the adoption of the DVCS “git” by the FreeBSD community it is now inexpensive to base your commercial project on FreeBSD and stay up to date.

The problem many people have faced is that they usually do not have the time nor expertise to manage “vendor imports” of a large project like FreeBSD.  The majority of companies have time/budget to hire a handful of competent developers and until the advent of DVCS it was extremely time consuming to set up any sort of system to track a large open source project such as FreeBSD.

This blog post details recent work I’ve done to bring a product under Git version control and thereby make it extremely easy for us to track FreeBSD changes while also maintaining our project’s history.

While most DVCS articles focus on upstreaming changes and typically only shows the workflow for a single developer, this article will instead focus on maintaining your private changes for the long term as well as creating long term branches for your team to develop against.

Getting started.

Install git one of these ways:

  • pkg install devel/git
  • pkg_add -r git
  • cd /usr/ports/devel/git && make all install

Setting up a mirror of FreeBSD’s git repository.

/ % mkdir ~/blog
/ % cd ~blog
# this next step takes about 30 minutes to run on my fast computer with fast networking
~/blog % git clone https://github.com/freebsd/freebsd.git
# move this clone to the side, it has the correct remote references, but is not a "bare" repo which is needed 
# for a centralized repo that will be shared with a team
~/blog % mv freebsd badclone
# now make the actual bare repo
~/blog % git clone --bare --shared badclone freebsd.git

This will take a while.  Expect an hour or so.

Now we need to update the git config file for your bare repo to point to the remote repo.

edit the following file:

~/blog % vi ~/blog/freebsd.git/config

Change the section under [remote “origin”] from:

[core]
        repositoryformatversion = 0
        filemode = true
        bare = true
[remote "origin"]
        url = /home/alfred/blog/badclone

to:

[core]
        repositoryformatversion = 0
        filemode = true
        bare = true
[remote "origin"]
        url = https://github.com/freebsd/freebsd.git
        fetch = +refs/heads/*:refs/remotes/origin/*

Now get that branch we’re going to base our work on:

~/blog % cd freebsd.git
# this command will take a few minutes
~/blog/freebsd.git % git fetch
~/blog/freebsd.git % git fetch 
From https://github.com/freebsd/freebsd
 * [new branch]      master     -> origin/master
 * [new branch]      projects/altix -> origin/projects/altix
 * [new branch]      projects/altix2 -> origin/projects/altix2
 * [new branch]      projects/amd64_xen_pv -> origin/projects/amd64_xen_pv
 * [new branch]      projects/arm_eabi -> origin/projects/arm_eabi
 * [new branch]      projects/armv6 -> origin/projects/armv6
 * [new branch]      projects/arpv2_merge_1 -> origin/projects/arpv2_merge_1
...[snip]... 
 * [new branch]      projects/geom_raid5 -> origin/projects/geom_raid5
 * [new branch]      projects/graid/7 -> origin/projects/graid/7
 * [new branch]      projects/graid/8 -> origin/projects/graid/8
 * [new branch]      projects/graid/head -> origin/projects/graid/head
 * [new branch]      projects/gvinum -> origin/projects/gvinum
...[snip]...
 * [new branch]      release/4.1.0 -> origin/release/4.1.0
 * [new branch]      release/4.1.1 -> origin/release/4.1.1
 * [new branch]      release/4.10.0 -> origin/release/4.10.0
 ...[snip]...
 * [new branch]      releng/8.3 -> origin/releng/8.3
 * [new branch]      releng/9.0 -> origin/releng/9.0
 * [new branch]      releng/9.1 -> origin/releng/9.1
 * [new branch]      stable/2.0.5 -> origin/stable/2.0.5
 * [new branch]      stable/2.1 -> origin/stable/2.1
 * [new branch]      stable/2.2 -> origin/stable/2.2
 * [new branch]      stable/3   -> origin/stable/3
 * [new branch]      stable/4   -> origin/stable/4
 * [new branch]      stable/5   -> origin/stable/5
 * [new branch]      stable/6   -> origin/stable/6
 * [new branch]      stable/7   -> origin/stable/7
 * [new branch]      stable/8   -> origin/stable/8
 * [new branch]      stable/9   -> origin/stable/9
 * [new branch]      svn_head   -> origin/svn_head
...[snip]...
 * [new branch]      user/alfred/9-alfred -> origin/user/alfred/9-alfred
 * [new branch]      user/alfred/so_discard -> origin/user/alfred/so_discard
~blog/freebsd.git % git branch --track releng/9.0 origin/releng/9.0
~blog/freebsd.git % git fetch origin releng/9.0
# we're now done with "badclone" so kill it.
~blog/freebsd.git % rm -rf ~/blog/badclone

Now make a working clone to branch for our project:

We are going to make a branch called “myproject-9.0″ that is based off of the “releng/9.0″ FreeBSD branch.

We make a private clone and then create the branches we need.

~blog/freebsd.git % cd ~/blog
# make a private clone
~/blog % git clone -b releng/9.0 ~/blog/freebsd.git myproject
~/blog % cd myproject
~/blog/myproject % git branch                            
* releng/9.0
# make our branch
~/blog/myproject % git checkout -b myproject-9.0 releng/9.0
Switched to a new branch 'myproject-9.0'
# share this branch in the repository in /git-repo/freebsd.git
~/blog/myproject % git push origin myproject-9.0

You should now be able to “git clone -b myproject-9.0 /git-repo/freebsd.git” directly from our “bare” repo!  This is just testing that so far we’ve done everything right.

~/blog % git clone -b myproject-9.0 ~/blog/freebsd.git ~/blog/myproject-9.0
Cloning into '/home/alfred/blog/myproject-9.0'...
done.
Checking out files: 100% (49061/49061), done.
~/blog % rm -rf ~/blog/myproject-9.0

So now what?

Well your team goes off to work on the myproject-9.0 branch from /git-repo/freebsd-git.

A few months go by and now you want to migrate your project to FreeBSD 9.1.

How do we do this?

We create a copy branch and then rebase it onto FreeBSD 9.1.

First we set our main repo to track the new 9.1 branch:

% cd ~/blog/freebsd.git
~/blog/freebsd.git % git fetch
~/blog/freebsd.git % git fetch origin releng/9.1
From https://github.com/freebsd/freebsd
 * branch            releng/9.1 -> FETCH_HEAD
~/blog/freebsd.git % git branch --track releng/9.1 origin/releng/9.1
Branch releng/9.1 set up to track remote branch releng/9.1 from origin.

Now we can go into our private repo, create a “copy branch” then rebase it onto FreeBSD 9.1:

% cd ~/blog/myproject
# mirror FreeBSD 9.1
~/blog/myproject % git fetch
From /home/alfred/blog/freebsd
 * [new branch]      releng/9.1 -> origin/releng/9.1
~/blog/myproject % git checkout releng/9.1
Checking out files: 100% (10203/10203), done.
Branch releng/9.1 set up to track remote branch releng/9.1 from origin.
Switched to a new branch 'releng/9.1'
# copy my project branch, note after this command, both myproject-9.1 and myproject-9.0 are based on 9.0
~/blog/myproject % git checkout -b myproject-9.1 myproject-9.0
# now rebase this new branch (myproject-9.1) onto FreeBSD 9.1
~/blog/myproject % git rebase --onto releng/9.1 remotes/origin/releng/9.0 myproject-9.1

This will put you into a git rebase.

Once you are complete with your rebase your project branch ‘myproject-9.1′ should be based on FreeBSD 9.1.

I strongly suggest you use a visual tool such as “gitk” to check your work, go through each diff and watch for nasty stuff.

you should then push it to your shared repo:

git push -f origin myproject-9.1

Enjoy!

Soon I will cover tracking a moving target such as FreeBSD 9-stable.

About these ads

From → Uncategorized

4 Comments
  1. Warner Losh permalink

    Would love to see a ‘how to mange your work flows to contribute back a subset of your customizations’ section as well…

    • I’ll be adding it. I think what will have to happen is that the developer will have to setup a git-svn and pull branches from this repo. That is until FreeBSD finally switches to git.

      So developer will have a clone with remotes to the “commercial main repo” as well as “git-svn” against FreeBSD proper. Using that one should be able to cherry-pick from “commercial main” into “git-svn” and then “dcommit” back to FreeBSD.

      I’ll update. As I get there.

  2. Uli permalink

    You should just git clone –bare the first repo, saving precious time.

    The github repos also have a svn_head branch, which is a git-svn conversion of the tree, use this as described in the wiki to bootstrap your own git-svn clone (starting from scratch easily takes 4-5 days)

  3. Outbackdingo permalink

    not sure when this was written but, if you delete badclone as instructed, the following
    # make a private clone
    ~/blog % git clone -b releng/9.0 ~/blog/freebsd.git myproject

    fails.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: