Automated unit tests are hard to write. Software architecture must be designed carefully to allow unit testing. You have to spend time to write tests as well and it’s not easy to write good tests. It’s easy to make big mess that is not easy to maintain after few weeks.
On the other hand automated integration tests are hard to maintain and are fragile. You can “write” them pretty easy in a record-and-replay tool, but later they show their real cost during maintenance.
But there’s an answer for problems mentioned above. Do you know Eiffel language? The language has special built-in constructs that support executable contract specification. It’s called Design By Contract (DBC). DBC is easier to write and to maintain because no special test cases need to be specified, just conditions for method parameters, expected results and state invariant that must be preserved. How DBC can substitute old-fashioned tests? Read on!
(more…)
When you have auto-build and auto-test process already in place you can use the same infrastructure to catch early performance problems as well. It’s not as complicated as you may think.
First of all you can monitor your CPU/IO usage during tests and take snapshots on “errant” situation. If there’s no high local processing probably 100% CPU usage means that there are some performance problems in your software. Sample:
top -b -n 1 | awk '
/ 0% idle/ { enable=1 }
$7 > 20 && $0 && enable { print "kill -SIGUSR2 " $1; }
' | sh
Above script checks for 0% idle time and sends every process that uses above 20% of CPU SIGUSR2 signal. Installed signal handler will make snapshot of current running thread and will give enough information to fix the performance issue.
(more…)
A quite typical picture: software development company X, first delivery of project Y to testing team after few months of coding just failed because software is so buggy and crashes so often. Project Manager decides to invest some money in automated testing tool that will solve all stability problems with click-and-replay interface. Demos are very impressive. The tool was integrated and a bunch of tests were “clicked”.
After a month we have 10% of tests that are failing. 10% is not a big deal, we can live with them. After additional month 30% of tests fails because important screen design was changed and some tests cannot authorize them for some reason. Pressure for next delivery increases, chances to delegate some testers to fix failing test cases are smaller every week.
What are the final results of above tool?
- unmaintained set of tests (and the tool itself) is abandoned
- man-days lost for setting up test cases
- $$ lost for the tool and training
Has the tool been introduced too late? Maybe wrong tool was selected?
In my opinion automation and integration tests don’t play well together. Let’s review then main enemies of automation in integration tests:
Locate duplicates quickly: I mean only size+filename check, not expensive MD5 sum computation:
find . -printf "%f:%s:%p\n" -type f | \
awk -F: '
{
key=$1 " " $2;
occur[key]++;
loc[key]=loc[key] $3 " "
}
END {
for(key in occur) {
if(occur[key]>1) {
print key ": " loc[key]
}
}
}
' | sort
A bit of explanation of above magic:
- printf: tells find command to output file metadata instead of only file path (the default), this metadata (size, filename) will be used later
- -F: :We want to handle properly paths with spaces, that’s why special separator is used
- key=$1 ” ” $2: we use file name (without dir) and file size to create ID for this file
- occur: table (key -> number of file occurences)
- loc: maps file ID to list of locations found
- occur[key]>1: we want to show only files that have duplicates
- sort: results are sorted alphabetically for easier navigation
Web2py is “full stack” Python Web Framework, Lighttpd is fast, multi-threaded HTTP server. I’ll present a method to connect web2py-based application under lighttpd.
I assume the following setup is already done:
- A domain named “myapp.com” is configured to point to your server
- Python / lighttpd is already installed on server
- Your web2py app is placed under /var/www/web2py
- Your web2py app has application “myapproot” configured
First of all, you have to configure lighttpd to locate web2py application, create file /etc/lighttpd/conf-enabled/myapp.conf:
(more…)
