Dariusz on Software

Yesterday SMS notifications were added to site-uptime.net. You can specify cell phone number and get information on network problems directly to your cell phone. I hope this option will be useful for S-U customers (existing and new).

Correcting comments of existing commits is easy if you have enough privileges on subversion repository:

svn propedit -r <revision-number-here> --revprop svn:log "<new log message>" <url>

That's all. Pretty simple.

site-uptime.net, www sited monitoring tool has been just extended with feature to optionally delay notification until problems is confirmed during next (or third) check. It's useful if you encounter with random short network failures and want to skip notification on such short living problems.

However, such random problems may show efficiency problems with your site (timeouts). You should check carefully what's hidden behind them (web server error log, if available).

If you are administering small-memory VPS servers it's very easy to exceed all available memory. Typical memory hogs (apache mpm-prefork, rsyslogd) could be easily replaced by alternatives. It's not very easy to do with packaging system (you have to be up to date with security updates).

I compared memory usage of two APT interfaces: apt-get and (new, now prefferred) aptitude. Here are the results (top output when command shows list of packages to install):

# apt-get install munin
----------------------------
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
28800 root      20   0 15940  13m  10m T    0 10.3   0:00.44 apt-get

# aptitude install munin
----------------------------
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
28822 root      20   0 40936  33m  12m T    0 26.3   0:00.92 aptitude

As you can see aptitude uses almost 3x more memory than apt-get (VSZ and RSZ). If you are low on memory on low-end box it's noteworthly saving.

Note: tests were done on 32-bit OpenVZ-based VPS.

I've been using many version control systems in software development projects. I'm aware Subversion is not the best version control system out there, but it's most popular (at least among enterprise projects I've been working on).

I'm describing here Subversion usage best practices that come from my experience with Subversion (or other version control systems) collected on many projects. Some suggestions are related strictly to Subversion, some are general.

Commit complete, logical changesets

Always commit complete patch. "Complete" means the commit:

• shouldn't have compilation errors
• shouldn't lower successful test cases counter
• should have precise purpose (one fixed bug, implemented feature, as so on...)

Why complete changesets are so important? You can easily then:

• reapply only selected bugfixes on another branch (sometimes it's called cherry-picking)
• undo change that made system unstable by simple reverted merge

Sometimes you have many different changes (example: two bugfixes, one feature added) in your workspace and it seems the easiest way is to commit them in one big commit. DON'T. You can use "svn diff", "patch" commands to split your local changeset into few parts and commit them separately. How? It's simple (but not obvious at first glance):

1. svn diff > ../changeset.diff
2. $EDITOR ../changeset.diff
3. Leave only changes to be removed from working copy, delete remaining changes
4. patch -p0 -R <  ../changeset.diff
5. Now you have yor workspace without changes to be removed, retest, commit
6. patch -p0 <  ../changeset.diff
7. Now you have only changed that previously were skipped, retest, commit

Special tool have been implemented that can automate above flow: PMS (Patch Management System).

Do not commit workspace partially

If you commit only subset of files from your working copy you risk breaking the build. The proper way for splitting working copy into separate commits is to reverse patch your changeset (see previous practice examples).

Similar problem appears when you forget to add to versioned files new file (and it appears as "?" on "svn status" listing). Use "svn status" frequently and properly set svn:ignored where necessary.

Continuous integration simplified: commit often, update very often

If you postpone your changes more that two days it's very likely someone will use obsolete API that you're refactoring and someone will get compilation errors. Try to make atomic but small commit to minimize such risk.

When you develop a feature it's very likely you will use obsolete API that some other developer is refactoring now. Update very often to see fresh system state.

Have you noticed above paragraphs are very similar? You have to make fast information flow. Making long-living branches will postpone new code from being integrated. Frequent merges and commits are implementation of continuous integration. Every integration problem (as API change mentioned in this section) will be visible as soon as possible (when it is stable enough to be committed).

Someone can point out that frequent commits can destroy trunk stability - see next section for answer.

Shared branches must be stable, Really

When you are working on isolated topic branch you don't need to ensure it's stable enough to be committed. It's not needed to compiler, either. But if anyone will have access with you to this branch/trunk you must ensure it will remain stable. Stable branch encourage frequent updates and it's base to Continuous integration.

How can we define "stable" criteria for local changeset to be committed:

• Code compiles with no errors
• No additional failing tests (no regression!)
• Ensure no accidental whole-file reformatting by viewing diffs (svn di)
• Ensure no forgotten unversioned files (svn status = ?) in your repository (it may break compilation for someone else)

Use meaningful commit comments

A typical error (yes, error) made by novice programmers is empty or "asdf"-style commit comments. The information what was committed is hidden in commit changeset only. It will make code inspection harder. I suggest to enter at least in commit comment:

• Issue tracker ID if preset (to link commit to issue fixed)
• short one-line description (can be copied from issue tracker)

Then anyone can look at fresh commits:

svn log -l 10

and see what's going on now in the project.

Care about merge and annotate - DO NOT reformat whole file

Sometimes when I want to trace who are responsible for particular change in file I'm discovering by annotate that whole file have been changed recently by one person. Why? This programmer reformatted whole file and committed big changeset. Now file looks better but at what price? Original committer information is lost.

Similar problem occurs when you merge such changeset (from topic branch to trunk for instance). There are many conflicts because merge changeset overlap with changesets committed previously on trunk.

Subversion like many other popular version control systems loses commit information on merge (merged person is visible instead of original authors of merged changesets). If you expect to preserve such information use GIT. (Thanks Michal to pointing this out: Subversion >= 1.5 tracks merges).

Setup auto properties correctly

In mixed systems environment there are many formats of storing files. EOL (end of line) standards differs between operating systems. Subversion allows to convert-on-fly EOL style. In order to make use of such feature you have to set "svn:mime-type: text/plain".

Setting such properties on every added files can be boring, so auto properties have been introduced in Subversion. You can map (locally) properties to extensions in your ~/.subversion/config:

[auto-props]
*.csv = svn:mime-type=text/plain
*.java = svn:mime-type=text/plain
*.sql = svn:mime-type=text/plain

Setup svn:ignore in repository

Some files are required to live in project directory but shouldn't be committed into SVN (*.class, *.ear, ...). General "ignore" mechanism marks such files (or file masks) to be invisible for SVN (not reported as unknown files "?"). Typically CVS used .cvsignore file, Bazaar reads .bzrignore, SVN reads svn:ignore property AND local user preferences stored in ~/.subversion/config.

Do not depend on local developer ignore settings in ~/.subversion/config. Attach svn:ignore tag to proper directories to be present on every working copy. It will make to add new filters easier (by commiting new properties once to Subversion).

During a merge from branch on the same repository as current copy:

svn merge -r17007:17249 http://hostname/svn/path

I got the error: svn: Malformed URL for repository. The fix is to add user name in SVN URL:

svn merge -r17007:17249 http://dcieslak@hostname/svn/path

Is this a Subversion bug? Probably the problem is that my username placed in SVN config file has form: domainname\username. Maybe SVN merge cannot handle that format?

Linode, VPS hosting company I'm using, just opened signups to new dataceneter in Europe. This new deployment is located at Telecity Group’s Powergate facility.

Checking tracerote to London from Warsas shows the benefits:

Packets               Pings
 Host                        Loss%   Snt   Last   Avg  Best  Wrst StDev
 (...)
 5. war-b2-link.telia.net     0.0%    21   32.7  32.9  31.7  39.8   1.7
 6. ffm-bb1-link.telia.net    0.0%    21   32.7  34.3  30.3  83.4  11.4
 7. prs-bb1-link.telia.net    0.0%    21   40.7  46.6  40.7 100.3  14.7
 8. ldn-bb1-link.telia.net    0.0%    21   48.9  55.1  48.0 127.5  20.5
 9. ldn-b2-link.telia.net     0.0%    21   48.5  48.7  48.0  49.7   0.4
10. globix-118206-ldn-b2.c.t  9.5%    21   48.8  58.0  48.3 216.9  38.5
11. te4-1-dist65-01.lon10.te  0.0%    21   50.0  49.5  48.6  50.1   0.4
12. london1.linode.com        0.0%    21   49.0  50.0  48.7  57.9   2.0

Now packet travels only 50ms to server from Warsaw (Newark location I'm using now has 130ms). It's big improvement.