• In-place branches (I used to use them heavily)
• Faster (no Python libs loading during “cold” start)
• Can’t live without “git rebase -i” now

• invalid name (wrong artifact number)
• invalid comment inside (also based on wrong artifact number)

it was a mistake and programmer wanted to preserve changes done on branch, but using different name.

The solution I proposed was to:

1. clone existing branch under different name
2. “amend” last commit on this new branch (to fix comment)
3. push new branch into correct location
4. drop old branch

Sequence of GIT commands was:

$ git branch corect-name origin/invalid-name
(We saved desired SHA with new name)
$ git checkout corect-name
$ git commit --amend
(Fix comment of last commit in editor)
$ git push origin corect-name
(Fixed branch published on our default remote)
$ git push origin :invalid-name
(Weird GIT syntax for deleting remote branches)

Remote branch in GIT is a head that tracks branch stored on server. You shouldn’t update (commit) directly that branch. You can update local branches instead then use “push” to publish changes from local branch to remote. Remote branch can be used to answer the following questions:

• is my local branch up to date regarding to server state?
• what changes (diff/log) were added to my local branch and aren’t submitted (push) yet?

In order to use properly remote branches you have to create maching local branch for every remote branch. Boring and error-prone task. Let’s automate it:!

git branch -a | awk \
    '/RELEASE/ { sub("remotes/origin/", "", $1); \
    print "git branch --track " $1 " origin/" $1 }' | sh

Above command imports all branches that contain string “RELEASE” (I assume we may be interested in checking release status).

git cherry-pick <SHA-COMMIT-ID>

Also SVN has simple merge mode that supports selecting of single commit:

svn merge -c <REV-NO> <URL>

What about Perforce? After checking Perforce documentation for merging I hit the following syntax for selecting subset of changelists to merge:

p4 integrate //depot/release/jam/...@30,@30 ...

That’s it: add the same change list id twice (separated by comma) to source URL and you will get cherry-pick!

[2011-04-20] Update: looks like sometimes @CL,@CL will not merge change properly, it’s better to use SVN-style revision range mode (@CL-1,@CL):

p4 integrate //depot/release/jam/...@29,@30 ...

Perforce has additional step to perform:

p4 resolve -af

that walks thru all changes from integrate step and allow to check for conflicts. Of course in the end you have to commit changes.

p4 submit

BTW. I (still) don’t see any benefit from using Perforce over well-known SVN. Anyone?

Releasing Software is not just packing latest version

The aims for releasing procedure designed:

• allow for testing window before release date
• have the possibility for examine released version to test for reported bugs
• possibility to manage existing releases (hot-fixing critical bugs)

Prepare Release Candidate Branch

We want to stabilize and test some snapshot of current development branch. That’s why I’m forking few days before release RC branch from “master” and switch (it will be used for hot-fixing):

git branch RELEASE_0.2.1_branch
git push origin RELEASE_0.2.1_branch
git co RELEASE_0.2.1_branch

I’m marking RC state (RC=Release Candidate) to be able to see changes done for released version:

git tag RELEASE_0.2.1_RC
git push --tags

Anyone familiar with advanced CVS usage will see similarities to tagging for CVS merge purposes. GIT tracks merges automatically, however marking branch starting point is a good idea.

As you can see I created simple naming convention schema to manage releases.

Prepare Release

Time after creating RC branch and before releasing from this branch is the time for testers to do their job and sweep out as many defects as possible. Fixes are added directly on release branch (we will port them to master later).

When tests are done we are preparing release and tag current version by RELEASE_0.2.1

git commit -a -m "version number changed (#XYZ)"
git tag RELEASE_0.2.1
git push --tags

By this tag we will be able to inspect exact version that was sent to our clients.

Prepare hotfix

Sometimes out safety net composed of automated test suite and testing team fails and we have to fix errors reported from production. That is the purpose for release candidate branch. First, switch to correct branch that supports hot-fixed release:

git branch --track RELEASE_0.2.1_branch origin/RELEASE_0.2.1_branch
==OR==: git co RELEASE_0.2.1_branch; git pull origin

Tag current version that goes in this hotfix by RELEASE_0.2.1_hotfix_YYYYMMDD:

git commit -a -m "version number changed (#XYZ)"
git tag RELEASE_0.2.1_hotfix_YYYYMMDD
git push --tags

As you might noticed naming convention is based on branch name. Thanks to this convention we can answer the following questions:

• what hot-fixes were prepared for release X?
• what is the latest hotfix for release X?
• what was delivered in latest hotifix of release X?
• etc.

Porting back changes from branch to master

Sometimes changes made on branch will be useful for next releases. You can easily merge them back to master:

$ git co master
$ git merge RELEASE_0.2.1
$ git push

GIT tracks what have been merged already so you can merge/cherry-pick in both directions.

Typical Usage

What changes were included in latest hotfix compared to previous one:

 $ git diff RELEASE_0.2.1_hotfix_date1 RELEASE_0.2.1_hotfix_date2

What changes were added in new release:

$ git log  RELEASE_0.2..RELEASE_0.2.1
$ git diff RELEASE_0.2..RELEASE_0.2.1

During watching my commit list using gitk I noticed many developers are accidentially merging their changes without so called “fast forward” (additional commit is created and the history is not linear). Why? The cause is that they are pulling changes from server AFTER local commit. A example commit tree taken from this worth-reading article:

The solution for this problem is to use “git pull –rebase” when downloading changes from repository. Existing local commits will be “rebased” (SHA-ids will change) and the history will be left linear. It looks much better:

“Rebasing” can be requested during pull by using this syntax:

git pull --rebase

I bet you will forget that after n-th commit That’s why GIT allows to make rebase default option. Just do for every branch you have (including “master”):

git config branch.master.rebase true
git config branch.branch_10.rebase true
...

And tell GIT to setup such rule for every new branch:

git config branch.autosetuprebase always

Of course you can disable automatic “rebasing” when needed:

git pull --no-rebase

Perforce is a commercial VCS that is similar to CVS (revisions per file) and SVN (global revisions of whole tree). It’s support for status command is very clumsy. Let’s check how we can emulate status with this tool using small script:

echo === extra files not tracked by Perforce ===
find . '!' -type d '!' -executable | p4 -x - have 2>&1 | grep 'not on client' | \
 sed '/\/moc_/d;/\.so/d;/\.o /d;/Makefile /d;/\.a /d'

echo === Current changelist ===
p4 changelist -o

As you can see we implemented “ignore” mechanism in above script (by sed filtering). “Extra files not tracked by Perforce” reflects “?”-status from CVS/SVN. “Current changelist” reflects “A,D,U”-statuses from CVS/SVN.

By using such script you can ensure your commit contains all files from your workspace, thus will be buildable on other developers’ machines.

There was one development branch and one integration branch. Development branch most of this time was in not “buildable” state (permanent build error), so no one was able to ensure changes are not breaking the build before commit until stabilisation. Integration branch was loaded using files copied from development branch when stable state was achieved (I know, lame). This integration style (allowing for non-buildable commits on development branch) blocks parallel integrations (one have to wait for stable state on development branch).

Are there better ways to organise such integration?

Shared Codebase Ownership

In this scenario every team has full write access to all modules and it’s possible to create single commit by team T1 that spans module boundaries: refactors module A (owned by T2) AND calling code from module B (owned by T1).

There will be only one “a must” rule: all codebase should be compilable after every commit. Modifications to “not owned” modules should be minimised (only to make code compilable), probably with some “FIXME” left in code.

Doing that operations in single commit allows you (1) to inspect that commit afterwards (2) retain “green build” property after every commit.

Having always-compilable head we will be able to schedule such operations in paralell. Nobody will be blocked (of course teams have to communicate/consult such cross-border changes before commit).

In this case applied changes are visible by diff-ing selected changesets.

Topic Branches

If globally shared codebase is not an idea to be considered we could use short-living branches (called sometimes “topic branches”) that get updates from both teams and after stabilisation they are merged back into main branch (and deleted).

Code on topic branch could be unstable, on development branch must be always stable.

Note that merging person should have write access to all “dev” branch (in this aspect this strategy is similar to previous one). I used to place at this point code review process.

In this case pending changes are visible by doing a diff on existing topic branch.

Moving “Stable” Tag

This strategy can be used for file-tracking systems like CVS or Perforce. For those VCS-es one can move tag to new version for subset of files (SVN, GIT, Bazaar will not allow that).

Person that created “stable” commit should create/move some tag to current revision.

$ p4 tag -l STABLE ...

Then anyone interested in stable state would sync to that state:

$ p4 sync ...@STABLE

Then I can fetch fresh (from head) files I’m going to change:

$ p4 sync P1/...

After adding fixes I’m committing the change:

$ p4 submit

and moving stable label for files I’ve changed (I must check if full build is green before):

$ p4 tag -l STABLE P1/...

In this case pending changes are visible by diff-ing STABLE..head revisions range.

Summary

If some independent activities cannot be performed in paralell it’s a bad sign. It means some artificial dependencies were introduced and it results in slower progress for project (caused by serialisation).

In this case missing “W” access was the cause for additional burden. Unstable main branch was a global semaphore that blocked everyone (at least in integration terms).

Sometimes you want to share locally created GIT branch with other developers (code review, collaborative development on branch, etc). You can do it easily by issuing a command:

$ git push origin local_branch_name

If you decide to drop remote branch just issue:

$ git push origin :local_branch_name

Happy GIT-ing!

Perforce

Checkout of full tree requires to download many gigabytes of data thus was not an option. I was interested only in small part of whole repository. Subtree checkout syntax is not very obvious (had to google for it):

p4 sync //path/to/directory/...

I think, this hint may be useful to someone that must (want?)* use this VCS implementation.

(*) Who will pay 700 USD per user for Subversion replacement? Am I missing something valuable in P4 that is worth of this money? Commercial support ?

If you created your repo copy by “clone” operation you will have “origin” remote branch defined. This remote can be used to pull/push changes.

$ git remote -v
origin zeus.aplikacja.info:cust-proj1

If you decide to change this definition later you can issue the following commands:

$ git remote rm origin
$ git remote add origin git@github.com:aplikacjainfo/proj1.git
$ git config master.remote origin
$ git config master.merge refs/heads/master

After this change you can push your commits to new repository location (origin is selected as default remote branch for master, it’s configured in .git/config):

$ git push

That’s all. Much simpler than moving Subversion repository.

UPDATE 2011-12-09: replaced sed command with much simpler “git config” replacement.

Fortunately current version of GIT supports that nice feature. It can be enabled with few settings in ~/.gitconfig file:

[diff]
color = true

[pager]
color = true

[status]
color = true

Additionally pager used (less in my case) should support ANSI colors (~/.bash_profile):

export LESS="-R"

And now diffs are rendered using colors that improve readability (anyone who doesn’t review changesets before commit? ). Now diffs look much better on a X terminal:

svn merge -c -19203 https://REPO_URL

In example above you do reverse merge of 19203 revision (note “-” sign before revision number). After that merge:

• Inspect if workspace compiles without errors
• If it’s OK: commit local changeset
• Notify 19203 committer about the change

In order to fix broken commit original author should do the opposite:

svn merge -c 19203 https://REPO_URL

(note there’s no “-” before revision number). Then:

• Correct changeset and test if workspace is not broken
• If it’s OK: commit local changeset (it will be nice to show original revision number in a comment)

The property I’m going to present today is “mime-type“. It describes file content in similar way to HTTP header “Content-type” telling svn client how to handle the file. Typical values are: “text/plain”, “application/octet-stream”. Especially first part of mime-type is important:

• text/*: line-by-line merges are used, diffs are generated
• any other prefix: no text merges prepared

If you do not set this properly right you may end up with messed binary file (end-of-line conversions) or non-mergable changes in text file (that is marked as binary by mistake).

Of course during adding a file to workspace one can forget to set those properties correctly. Here auto-props comes with help. Auto-props are applied when “svn add” command (from command line or from GUI) is issued. Configuration is placed inside “~/.subversion/config” file. Here’s my config fragment from one of projects.

[auto-props]
*.csv = svn:mime-type=text/plain
*.java = svn:mime-type=text/plain
*.sql = svn:mime-type=text/plain
*.sql = svn:keywords=Author Date Id Revision URL
*.jar = svn:mime-type=application/octet-stream

Besides mime-type svn:keywords is being set in the example. It controls which keywords are expanded in source files.

Questions

Important knowledge that may be required by developers doing updates may be summarized in few sentences:

• Who changed recently that line of code?
• When this method have been changed?
• Why algorithm works that way?

There’s simple method of automatically saving and retrieving this kind of information: Subversion (or any other version control system). How?

Answers

There’s nice feature of version control system that is not the most frequent used but is very useful: annotation/blame. This special view shows you for a file:

• Who changed this line?
• When this line was changed?
• Revision number of commit => Log entry => Bug tracker task ID => rationale (Why)

After locating such information you may have better understanding of source code.

How to check annotation using different tools:

• svn annotate filename
• git annotate filename
• bzr annotate filename
• Eclipse: Team / Show Annotation

The problem

Looks simple, but there’s a “quirk” here. If you are doing massive code changes (to enforce n+1th coding standard) you are overwriting original source code authors and information. Thus annotation (and log) becomes useless.

That’s why I’m asking you:

Do not reformat whole files on commit, PLEASE!

• any commit done to trunk is protected by set of automated tests that should run 100% clean (more critical project will use framework for a continuous build process like Cruise Control or PQM).
• “risky” tasks are done using peer code review process based on version control software

What to review?

We believe the most efficient method of reviewing code is to look at changesets (unified diffs). Then reviewer can see exact changes done by another developer to meet task criteria and codebase to review is minimized to really important parts of code (changes with some lines of context). That’s why we believe forming proper changesets is very important requirement.

Commit methods

There are mainly three methods of inserting new commits into trunk (specified in our implementation by special “target” field in bug tracker):

• Direct commit (target=master) without code review: used mainly for simplest functionality without much risk involved. Minimal administration burden, minimal merge conflict.
• Patch Based Review (target=patch) using PMS functionality for one-way code review: in this scenario only one developer is supposed to modify patch, another one is only reviewing. At least two developers are involved, merge conflict minimized by basing on trunk (patch is always re-based on trunk during development).
• Shared branch (target=branch) using VCS: Development is branched on separate branch and reviewved on this branch. Many developers can commit on this branch/review before merge. There is higher merge conflict risk for long development on branch.

A convention used by us: patch / branch name should contain noun from task summary and task id separated by a dash:

noun-task_id

for example: invoice-2351. This way we can easily find topic branches and precisely point to taks specification (by a task id).

Peer Code Review

Where’s peer code review here? Three modes of commit, three review configurations follows:

• Direct commit: changeset is reviewed after commit on trunk (as I said lower risk encourages us to be flexible)
• Patch-based (read-only) changeset view: patch is sent to another team member for review by email or shared patches repository
• Classical “topic branch”: read/write review: team member switches to topic branch and checks latest commits, he may also place comments in source code

Old, Good TODOs

What is a result of code review? The most efficient way to track problems found is to insert comments inside source code (in comments, near problematic code location). We prepared a convention of well-known TODO tags in source code: TODO:nick description. “nick” is abbreviation of comment recipient, description should be short and valuable. Here’s the example:

/*
TODO:DCI VAT report summary should show taxes grouped by rates,
not the case now
*/

Recipient then scans all source code for such TODOs and fix problems found (thus removes TODO in the same commit). This way:

• we can connect problem report with problem fix in one changeset
• we can track who entered comment and who fixed it (from SVN history)

Summary

Code inspections were found to be very effective method of raising quality of software (the IBM story). I believe informal implementation called peer review plays very well with other agile tools we are using for developing better software.

The possible reason for freezing commits:

• creating releases from trunk
• creating releases from branch
• commit freeze during merge from different branch
• move to different version control system
• massive changes expected that may cause many conflicts

Creating releases

If you want to make minor changes related to release and block any other (probably more risky) changes to be accidentially introduced you needn’t freeze commits. The more efficient solution here is to fork a branch. On separate branch you can do any justification you need to build the binaries for release.

Merging

For time-consuming merges (especially when many conflicts are present) it’s tempting to prevent commits on target branch to minimize problems related to local development during merged changes commit. I think merging person should perform frequent updates insted from current branch and match merged changes to current trunk state.

Switch version control software / repositories

Switching between version control system is a big change in development team. One has to learn new toolset to operate efficiently with new version control system. Postponing commits on old repository is not required. Those changes could be reapplied later by creating patch from missing changesets and applying them on new repository. Path format is a standard that allow to move changesets beteen different repositories.

Summary

In my opinion temporary blocking commits (so called “commit freeze”) is not a good idea. Agile methodology (the one we use at Aplikacja.info) requires frequent information sharing. There are alternatives that have lower impact on development and not get in the way for normal code flow.

svn propedit -r <revision-number-here> --revprop svn:log "<new log message>" <url>

That’s all. Pretty simple.

I’m describing here Subversion usage best practices that come from my experience with Subversion (or other version control systems) collected on many projects. Some suggestions are related strictly to Subversion, some are general.

Commit complete, logical changesets

Always commit complete patch. “Complete” means the commit:

• shouldn’t have compilation errors
• shouldn’t lower successful test cases counter
• should have precise purpose (one fixed bug, implemented feature, as so on…)

Why complete changesets are so important? You can easily then:

• reapply only selected bugfixes on another branch (sometimes it’s called cherry-picking)
• undo change that made system unstable by simple reverted merge

Sometimes you have many different changes (example: two bugfixes, one feature added) in your workspace and it seems the easiest way is to commit them in one big commit. DON’T. You can use “svn diff”, “patch” commands to split your local changeset into few parts and commit them separately. How? It’s simple (but not obvious at first glance):

1. svn diff > ../changeset.diff
2. $EDITOR ../changeset.diff
3. Leave only changes to be removed from working copy, delete remaining changes
4. patch -p0 -R <  ../changeset.diff
5. Now you have yor workspace without changes to be removed, retest, commit
6. patch -p0 <  ../changeset.diff
7. Now you have only changed that previously were skipped, retest, commit

Special tool have been implemented that can automate above flow: PMS (Patch Management System).

Do not commit workspace partially

If you commit only subset of files from your working copy you risk breaking the build. The proper way for splitting working copy into separate commits is to reverse patch your changeset (see previous practice examples).

Similar problem appears when you forget to add to versioned files new file (and it appears as “?” on “svn status” listing). Use “svn status” frequently and properly set svn:ignored where necessary.

Continuous integration simplified: commit often, update very often

If you postpone your changes more that two days it’s very likely someone will use obsolete API that you’re refactoring and someone will get compilation errors. Try to make atomic but small commit to minimize such risk.

When you develop a feature it’s very likely you will use obsolete API that some other developer is refactoring now. Update very often to see fresh system state.

Have you noticed above paragraphs are very similar? You have to make fast information flow. Making long-living branches will postpone new code from being integrated. Frequent merges and commits are implementation of continuous integration. Every integration problem (as API change mentioned in this section) will be visible as soon as possible (when it is stable enough to be committed).

Someone can point out that frequent commits can destroy trunk stability – see next section for answer.

Shared branches must be stable, Really

When you are working on isolated topic branch you don’t need to ensure it’s stable enough to be committed. It’s not needed to compiler, either. But if anyone will have access with you to this branch/trunk you must ensure it will remain stable. Stable branch encourage frequent updates and it’s base to Continuous integration.

How can we define “stable” criteria for local changeset to be committed:

• Code compiles with no errors
• No additional failing tests (no regression!)
• Ensure no accidental whole-file reformatting by viewing diffs (svn di)
• Ensure no forgotten unversioned files (svn status = ?) in your repository (it may break compilation for someone else)

Use meaningful commit comments

A typical error (yes, error) made by novice programmers is empty or “asdf”-style commit comments. The information what was committed is hidden in commit changeset only. It will make code inspection harder. I suggest to enter at least in commit comment:

• Issue tracker ID if preset (to link commit to issue fixed)
• short one-line description (can be copied from issue tracker)

Then anyone can look at fresh commits:

svn log -l 10

and see what’s going on now in the project.

Care about merge and annotate – DO NOT reformat whole file

Sometimes when I want to trace who are responsible for particular change in file I’m discovering by annotate that whole file have been changed recently by one person. Why? This programmer reformatted whole file and committed big changeset. Now file looks better but at what price? Original committer information is lost.

Similar problem occurs when you merge such changeset (from topic branch to trunk for instance). There are many conflicts because merge changeset overlap with changesets committed previously on trunk.

Subversion like many other popular version control systems loses commit information on merge (merged person is visible instead of original authors of merged changesets). If you expect to preserve such information use GIT. (Thanks Michal to pointing this out: Subversion >= 1.5 tracks merges).

Setup auto properties correctly

In mixed systems environment there are many formats of storing files. EOL (end of line) standards differs between operating systems. Subversion allows to convert-on-fly EOL style. In order to make use of such feature you have to set “svn:mime-type: text/plain“.

Setting such properties on every added files can be boring, so auto properties have been introduced in Subversion. You can map (locally) properties to extensions in your ~/.subversion/config:

[auto-props]
*.csv = svn:mime-type=text/plain
*.java = svn:mime-type=text/plain
*.sql = svn:mime-type=text/plain

Setup svn:ignore in repository

Some files are required to live in project directory but shouldn’t be committed into SVN (*.class, *.ear, …). General “ignore” mechanism marks such files (or file masks) to be invisible for SVN (not reported as unknown files “?”). Typically CVS used .cvsignore file, Bazaar reads .bzrignore, SVN reads svn:ignore property AND local user preferences stored in ~/.subversion/config.

Do not depend on local developer ignore settings in ~/.subversion/config. Attach svn:ignore tag to proper directories to be present on every working copy. It will make to add new filters easier (by commiting new properties once to Subversion).

svn merge -r17007:17249 http://hostname/svn/path

I got the error: svn: Malformed URL for repository. The fix is to add user name in SVN URL:

svn merge -r17007:17249 http://dcieslak@hostname/svn/path

Is this a Subversion bug? Probably the problem is that my username placed in SVN config file has form: domainname\username. Maybe SVN merge cannot handle that format?

We (Aplikacja.info) are a small Polish software company that write mostly in Python language and use Linux for development. GIT is an advanced version control system that was created by Linux Torvalds for maintain Linux kernel source tree. I’ll show how git can be connected in Unix shell environment with make tool.

Why GIT have been chosen to support our version control needs?

• Failure in development server will not block our work
• Access to history is very fast (it’s stored on every working copy)
• I used to work during flight, GIT supports all VCS operations off-line (commit, diff, etc.)

The hard thing about GIT is that it may confuse beginners. If you are one, remember:

• Always use git diff HEAD instead of git diff
• Always using git commit -a instead of git commit

Some of this gotchas have been addressed by some GIT interfaces, but they are deprecated now.

In order to ease GIT (and earlier, CVS) usage we introduced additional Makefile targets that handle typical source control tasks.

Examining working copy.

make st shows current working copy state with all local branches and current selected branch:

    st:
        git branch
        git status -a | cat

(cat simply remove pager call). make di shows differences between last committed version and working copy:

    di:
        git diff HEAD

Locally stored history (few last commits) can be inspected by make ch:

    ch:
        git log --pretty=oneline -15 | cat

We are using topic branches strategy to develop software, so switching branch (make sb) is a very common operation:

    sb:
        @read -p "name of existing branch to switch to [a-z_0-9]+: "\
            branch_name;\
        git checkout $$branch_name;

Local commits

If we have some changes uncommitted in working copy we can commit them now:

    commit: di st
        git commit -a

Above command shows current changes to be committed (di) and state of a repository (st) then allows to enter comment.

Going remote

Lets synchronise our working copy with central repo (without locally unmodified changes): make sync:

    sync:
        -git pull $(REPO) +`git branch | awk '/^\*/ {print $$2}'`
        git push $(REPO) `git branch | awk '/^\*/ {print $$2}'`

It downloads current branch from central server (this operation can fail when no branch exists yet, so we ignore errors here by “-” character) and pushes un-synced commits made.