On the other hand automated integration tests are hard to maintain and are fragile. You can “write” them pretty easy in a record-and-replay tool, but later they show their real cost during maintenance.

But there’s an answer for problems mentioned above. Do you know Eiffel language? The language has special built-in constructs that support executable contract specification. It’s called Design By Contract (DBC). DBC is easier to write and to maintain because no special test cases need to be specified, just conditions for method parameters, expected results and state invariant that must be preserved. How DBC can substitute old-fashioned tests? Read on!

General concepts for DBC are:

• pre-condition: you can specify rules for every input parameter to check contract fulfillment from caller side. If a pre-condition is not met that means there’s something wrong with caller
• post-condition: rules for post-method execution state can be specified as well. You can refer to input parameters, current object state and original object state (before method call)
• class invariants: refers to object members and must be valid after every public class call

As an example I’ll use DBC implementation called “pycontract” (Python language). It’s pretty easy to introduce in a project and uses standard doc-strings (like doctest module). No preprocessing is needed.

Pre-condition is specified by pre keyword:

pre:
    someObject != None and someObject.property > 0

Post-condition syntax uses some special syntax to allow to refer to old values, you have to mention mutable variables object that should be available for state change comparisions:

post[self]:
    self.property = __old__.self.property + 1

Above statement shows that property should be incremented by one after method return.

Invariant syntax is as follows:

inv:
    self.state in [OPEN, CLOSED, SENT]

All above statements must be embedded in method/class comments, here’s full example:

class SampleClass:
    """
    inv:
        self.state in [OPEN, CLOSED, SENT]
    """
    def open(self, x):
        """
        pre: x > 0
        post: self.state != CLOSED
        """

DBC assertions (like doc-strings) are disabled by default (they are just comments). You can decide to enable them for example in development builds (there’s an overhead related to DBC). An example how to enable pycontract:

import contract
contract.checkmod(modulename1)
contract.checkmod(modulename2)

Example failure (postcondition not met) caught by DBC:

DokumentFiskalny.testDokumentu() Traceback (most recent call last):
  File "test.py", line 164, in
    run_tests()
  File "test.py", line 144, in run_tests
    run_test_module(db, dao, moduleName)
  File "test.py", line 54, in run_test_module
    exec("%s.%s(env)" % (sName, sItem))
  File "", line 1, in
  File "", line 3, in __assert_testDokumentu_chk
  File "lib/contract.py", line 1064, in call_public_function_all
    return _call_all([func], func, va, ka)
  File "lib/contract.py", line 1233, in _call_all
    result = func.__assert_orig(*va, **ka)
  File "/home/darek/public_html/kffirma/DokumentFiskalny.py", line 433, in testDokumentu
    df.zapisz(args)
  File "/home/darek/public_html/kffirma/DokumentFiskalny.py", line 254, in zapisz
    args["idPodmiotuGospodarczego"],
  File "", line 3, in __assert_DokumentFiskalny__create_chk
  File "lib/contract.py", line 1165, in call_private_method_all
    return _method_call_all(getmro(cls), method, va, ka)
  File "lib/contract.py", line 1205, in _method_call_all
    return _call_all(a, func, va, ka)
  File "lib/contract.py", line 1241, in _call_all
    p(old, result, *va, **ka)
  File "", line 3, in __assert_DokumentFiskalny__create_post
  File "/home/darek/public_html/kffirma/DokumentFiskalny.py", line 155, in _create_post
    assert len(result) >= 2

The last part of this puzzle is coverage generation. DBC assertions are evaluated at runtime, so you have to ensure enough percentage code is execute during test runs. Im my experience you can rely on:

• integration-like scenarios (without explicit result verification)
• random input generation

For both cases you can integrate at high, UI level (keyboard, remote controller, HTTP, …).

Focus on quality techniques is crucial for any software project. If you (1) pay some effort in making your codebase resistant to regressions and (2) maintain that state during project lifetime you can gain more stable software. The real craft lies in selecting proper tools and having enough energy to implement them.

First of all you can monitor your CPU/IO usage during tests and take snapshots on “errant” situation. If there’s no high local processing probably 100% CPU usage means that there are some performance problems in your software. Sample:

top -b -n 1 | awk '
  / 0% idle/ { enable=1 }
  $7 > 20 && $0 && enable { print "kill -SIGUSR2 " $1; }
' | sh

Above script checks for 0% idle time and sends every process that uses above 20% of CPU SIGUSR2 signal. Installed signal handler will make snapshot of current running thread and will give enough information to fix the performance issue.

If your environment doesn’t allow to inspect current thread state at any time you can try to guess performance problems source and can introduce special profiling code to catch problems early. For example: typical performance problem is resource over-usage. Resource (remote server, database, …) may be called very often to retrieve the same data – it’s a candidate for local caching.

void ResourceProfiler::count() {
    if (!triggerCount) {
        return;
    }
    counter++;
    if (counter >= triggerCount) {
        time_t deltaS = time(NULL) - timeOfFirstOccurenceS;
        if (deltaS <= intervalS) {
            const int BUFFER_SIZE = 256;
            char buffer[BUFFER_SIZE];
            snprintf(buffer, BUFFER_SIZE, "Abused resource: %s=%d, %s=%ds", ENV_TRIGGER_COUNT, counter, ENV_INTERVAL_S, intervalS);
            warning(buffer);
        }
        timeOfFirstOccurenceS = 0;
        counter = 0;
    } else if (!timeOfFirstOccurenceS) {
        timeOfFirstOccurenceS = time(NULL);
    }
}

Above example shows core method of resource usage profiling code that issues a warning when there are more than triggerCount events in intervalS time frame. You can install this handler by calling count() when given resource is used (served is called, database record is loaded, …). Over-usage will be located dynamically during auto-tests.

Above method can be generalized to count usage per SQL query to locate very frequent SQL queries (typical N+1 SELECT problem). Instead of use one counter use Map<String, int> for counters and Map<String, time_t> for timestamps. It’s left as an exercise for a reader.

After a month we have 10% of tests that are failing. 10% is not a big deal, we can live with them. After additional month 30% of tests fails because important screen design was changed and some tests cannot authorize them for some reason. Pressure for next delivery increases, chances to delegate some testers to fix failing test cases are smaller every week.

What are the final results of above tool?

• unmaintained set of tests (and the tool itself) is abandoned
• man-days lost for setting up test cases
• $$ lost for the tool and training

Has the tool been introduced too late? Maybe wrong tool was selected?

In my opinion automation and integration tests don’t play well together. Let’s review then main enemies of automation in integration tests:

Initial state setup of environment and system itself

For unit-level tests you can easily control local environment by setting up mocks to isolate other parts of system. If you want test integration issues you have to face with integration-level complexity.

No more simple setups! If you want to setup state properly to get expected result you MUST explicitly set state whole environment. Sometimes it’s just impossible or extremely complicated to set it using UI. If you set states improperly (or just accept existing state as a starting point) you will end with random result changes that will make tests useless.

Result retrieval after operation

OK, we scheduled action A and want to check if record was updated in DB or not. In order to do that some list view is opened and record is located using search. Then record is opened and we can add expectations to that screen.

OK, but if we want to check if “e-mail was sent”? We cannot see that in application UI. Catching on SMTP level will be too unreliable and slow (timings).

It will just not work smoothly.

What’s next then?

It’s easy to criticize everything without proposing (and implementing) efficient alternatives.

My vision of integration testing:

• I’m not checking results for automated integration testing, it will just doesn’t work cannot be maintained efficiently
• I’m usually generating random input to cover as much functionality as possible using high level interfaces (UI)
• I depend on internal assertions/design by contract to catch problems during such random-based testing, they serve as an oracle (the more the results are better)

find . -printf "%f:%s:%p\n" -type f | \
    awk -F: '
        {
            key=$1 " " $2;
            occur[key]++;
            loc[key]=loc[key] $3 " "
        }
        END {
            for(key in occur) {
                if(occur[key]>1) {
                    print key ": " loc[key]
                }
            }
        }
    ' | sort

A bit of explanation of above magic:

• printf: tells find command to output file metadata instead of only file path (the default), this metadata (size, filename) will be used later
• -F: :We want to handle properly paths with spaces, that’s why special separator is used
• key=$1 ” ” $2: we use file name (without dir) and file size to create ID for this file
• occur: table (key -> number of file occurences)
• loc: maps file ID to list of locations found
• occur[key]>1: we want to show only files that have duplicates
• sort: results are sorted alphabetically for easier navigation

Web2py

I assume the following setup is already done:

• A domain named “myapp.com” is configured to point to your server
• Python / lighttpd is already installed on server
• Your web2py app is placed under /var/www/web2py
• Your web2py app has application “myapproot” configured

First of all, you have to configure lighttpd to locate web2py application,  create file /etc/lighttpd/conf-enabled/myapp.conf:

$HTTP["host"] =~ "(www\.)?myapp\.com" {
	server.indexfiles = ( "/myapproot" )
	server.document-root = "/var/www/myapp"
	server.dir-listing = "disable"
	fastcgi.server = (
		".fcgi" => ("localhost" => (
			"check-local" => "disable",
			"min-procs" => "1",
			"max-procs" => "2",
			"socket" => "/tmp/myapp.sock")
		)
	)

	url.rewrite-once = (
		"^/$" => "/ad",
		"^(/.+?/static/.+)$" => "/applications$1",
		"(^|/.*)$" => "/fcgihandler.fcgi$1",
	)
	$HTTP["url"] !~ "^(/ad|/fcgihandler.fcgi|/applications/myapproot/static/)" {
		url.access-deny = ("")
	}
}

Explanation:

• (www\.)?myapp\.com: regular expression to match domain with or without “www.” prefix
• server.indexfiles: specifies relative URL that should be called when only domain is given
• server.document-root: specifies location of web2py app in filesystem
• server.dir-listing: we do not want user to list our files using HTTP
• fastcgi.server: specifies where socket file is located
• url.rewrite-once: allow to use elegant (short) URLs
• url.access-deny: files other than static directory should be forbidden (security)

Then you have to configure fcgihandler.fcgi script properly:

(...)
fcgi.WSGIServer(application, bindAddress='/tmp/myapp.sock').run()

Note that /tmp/myapp.sock must be the same as specified in lighttpd configuration.

Then you have to start the fcgihandler.fcgi proces and ensure it will start on every boot. That’s all.

1. Developer creates and tests a feature
2. Before submitting commit to repository update/fetch/sync is done
3. Developer builds project again to check if build/basic functionality is not broken
4. Smoke tests
5. Submit

During step 3 you hear “damn slow rebuild!”. One discovers that synchronization with repository forces him to rebuild 20% of files in a project (and it takes time when project is really huge). Why?

The answer here is: header dependencies. Some header files are included (directly and indirectly) in many source code files, that’s rebuild of so many files is needed. You have the following options:

• Skip build dependencies and pray resulting build is stable / working at all
• Reduce header dependencies

I’ll explain second option.

The first thing to do is to locate problematic headers. Here’s a script that will find most problematic headers:

#!/bin/sh

awk -v F=$1 '
/^# *include/ {
    a=$0; sub(/[^<"]*[<"]/, "", a); sub(/[>"].*/, "", a); uses[a]++;
    f=FILENAME; sub(/.*\//, "", f); incl[a]=incl[a] f " ";
}

function compute_includes(f, located,
arr, n, i, sum) {
    # print "compute_includes(" f ")"
    if (f ~ /\.c/) {
        if (f in located) {
            return 0
        }
        else {
            located[f] = 1
            return 1
        }
    }
    if (!(f in incl)) {
        return 0
    }
    # print f "->" incl[f]
    n = split(incl[f], arr)
    sum = 0
    for (i=1; i<=n; i++) {
        if (f != arr[i]) {
            sum += compute_includes(arr[i], located)
        }
    }
    return sum
}

END {
    for (a in incl) {
        n = compute_includes(a, located)
        if (F) {
            if (F in located && a !~ /^Q/) {
                print n, a
            }
        }
        else {
            if (n && a !~ /^Q/) {
                print n, a
            }
        }
        for (b in located) {
            delete located[b]
        }
    };
}

' `find . -name \*.cpp -o -name \*.h -o -name \*.c` \
| sort -n

Sample output:

266 HiddenChannelsDefinitions.h
266 nmc-hal/hallogger.h
268 favoriteitemdefinitions.h
270 nmc-hal/playback.h
279 pvrsettingsitemdefinitions.h
279 subscriberinfoquerier.h
280 isubscriberinfoquerier.h
286 notset.h
292 asserts.h

As you can see there are header files that require ~300 source files to be rebuilt after change. You can start optimisations with those files.

If you locate headers to start with you can use the following techniques:

• Use forwad declaration (class XYZ) instead of header inclusion (#include “XYZ.h”) when possible
• Split large header files into smaller ones, rearrange includes
• Use PIMPL to split interfaces from implementations

The answer was located pretty quickly: socat tool can register to multicast sources and pipe output to many destinations (stdout for example). Let’s give a sample first:

socat UDP4-RECVFROM:9875,ip-add-membership=225.2.215.254:0.0.0.0,fork -\
 | grep -a '^s='

Values to change:

• 225.2.215.254: sample multicast IP address
• 9875: sample multicast port
• ^s=: string to be located in multicast data (in my case data protocol is textual)

Explanation:

• UDP4-RECVFROM: receive IPv4 UDP packets
• ip-add-membership: multicast requires registering of packets receiver
• 0.0.0.0: accept multicast on any client interface (you can specify here your client network address here as well)
• grep -a: sometimes you need just subset of incoming data, in my case textual data was mixed with binary (-a parameter for grep)

Waste of Over-production

Have you ever developed a module that was never used in final product? Or end users weren’t using some functionality because it was too complicated? You will probably say “yes” here (like me). How can we eliminate unneeded functionality from specs? How can we validate user interface to ensure a part of a system will be useful before actual build?

We have two opposite options here: prototyping with detailed design (yeah, Waterfall!), and iterative functionality delivery (bye, bye fixed-price contracts, welcome estimated iteration deliverables). The former is better for customer (“my budget is fixed”), the latter for development shop (lower risk of invalid estimates).

Waste of Defects

The Holly Grall of software development methodologies: to limit number of defects. There are many approaches here:

• Test-in quality at the end of a whole process (worst case IMHO)
• Parallel tests with development (waste of repeated tests)
• Automated system-level tests (may be hard to maintain!)
• Automated unit-level tests (my choose)
• Random DBC-based system level tests (promising …)

Defect means: slower delivery and it’s waste of resources (time, QA time, …).

Waste of Inventory

Means “Holding inventory (material and information) more than required”. Looks like it’s missing continuous integration practices. If you working on a topic too long time integration effort might me much bigger than syncing in smaller steps (minimum once a day?). The only problem with frequent syncing might me hitting regression (thus slowing work). But we can handle that by automated testing (see “Defect” above).

Waste of Over-Processing

“Processing more than required wherein a simple approach would have done” tells us we should prefer KISS (Keep It Simple Stupid) over heavy frameworks and complicated (fat) interfaces. In my opinion not the completeness of an interface designates it’s quality. Better interfaces are ones that are harder to be used in incorrect way (for example: eliminate default constructor if it does not have sense).

Waste of Transportation

“Movement of items more than required resulting in wasted efforts and energy and adding to cost”, in my field possible wastes are:

• duplicating bugtrackers within one project – needs syncing in both directions
• duplicating version control storage (local SVNs + central Perforce as a real example) – requires syncing methods
• submitting code always to more than one branch – is it better to cherry-pick set of commits from branch A to B then retest before submit?

Waste of Waiting

If team A is waiting for team B to produce module that will be used by team A it’s a waste of waiting. The solution:

• Create draft interfaces
• Stub implementations from lower layer

Then upper level modules can be started in parallel with lover level and stubs can be eliminated during integration. Interface will emerge as a result of two processes (A requirements + B possibilities).

Waste of Motion

How many “proxies” do we have for any information flow? Is there a person that just pushes some messages between customer and programmers without added value? Maybe we can make flows shorter (they will be faster and more effective).

Waste of Un-utilized People

Looks very similar to “Waste of waiting”. Our work strictly depends on people. We can improve our tools but people will be always be most important in software development activities. Avoid “push”-style, prefer “pull”-style work item assignment.

mkdir -p OLD; find . -maxdepth 1 -mtime +30 -exec mv \{\} OLD \;

Explanation:

• mkdir -p OLD: create OLD subdirectory if not exists
• -maxdepth 1: search only one level under current working directory
• -mtime +30: find only files that are older than 30 days
• -exec …. \;: for every file/directory found execute the following command
• \{\}: will be replaced by filename

Above command will move all old files/subdirectories into OLD subdirectory.

Web2Py

Static verification means locating simple bugs without running application thus very high (>95%) testing coverage (and high related cost) is not required. Instead with trying to cover by tests every possible screen/workflow/code line/… we can scan all codebase and search for some constraints. Most of them (based on my experience) are static – do not depend on runtime data thus can be effectively checked without running an application.

Here’s short example of web2py template language:

<a href="{{=URL('books','show')}}">...</a>

As you can see web2py will substitute {{=<python-expression>}}s by evaluated result. In this example URL points to existing module controllers/books.py and function inside this module named ’show’. I assume you see the problem here: one can select unknown module / function and it will result in runtime error.

First example is purely static: refence (URL(‘books’,’show’)) will not change during runtime, neither the source code. Then our static checker might be applied succesfully: check if all URL’s in all *.html files have proper functions defined in source code.

Technical solution can be composed to the following steps:

• locating all resources to check: scanning given directory in filesystem tree
• locating all interesting fragments in HTML file: I used regexp with arguments to easily extract interesting data
• locating functions in *.py code: because controllers are not plain python modules (expects some data in global namespace) I decided just to scan them textually

Another check that can be done is references inside HTML files (to CSS resources, JS files, …). This also can be automated:

<script type="text/javascript" src="svgcanvas.min.js"></script>

Source code refactorings might break your links/references and static scan might ensure you are not breaking application by refactoring.

Complete source code for URL() / SRC= / HREF=  checker:

import sys

sys.path.append("web2py/applications")
sys.path.append("web2py")

import os
import copy
import re
import string
from gluon.globals import *
from gluon.http import *
from gluon.html import *

RE_SRC = re.compile(r'src *= *"([^"]*)"')
RE_HREF = re.compile(r'href *= *"([^"]*)"')
RE_URL = re.compile(r'{{=URL\(\'([^\']*)\'')
RE_URL2 = re.compile(r'{{=URL\(\'([^\']*)\' *, *\'([^\']*)\'')

FILENAME= None
FNR = 0

request = Request()

def report_error(s):

    print "%s:%d: %s" % (FILENAME, FNR, s)

def check_src_exists(arg):

    if arg[0] == "{":
        # variable value
        return

    elif arg[0].find("{{"):
        # skip this URL
        pass

    elif arg[0] == "/":
        # absolute file
        fullPath = "web2py/applications" + "/" + arg
        if not os.path.exists(fullPath):
            report_error("file %s doesn't exists" % fullPath)

    else:
        # relative file
        fullPath = os.path.dirname(FILENAME) + "/" + arg
        if not os.path.exists(fullPath):
            report_error("file %s doesn't exists" % fullPath)

def check_href_exists(arg):

    #print arg
    if arg.startswith("{{="):
        pass
    elif arg.find("{{") > 0:
        pass
    elif arg.startswith("/"):
        if not os.path.exists("web2py/applications" + arg):
            report_error("absolute file %s doesn't exists" % arg)
    elif arg.startswith("http://"):
        # external link, do not check
        pass
    elif arg.startswith("https://"):
        # external link, do not check
        pass
    elif arg.startswith("mailto:"):
        # external link, do not check
        pass
    elif arg.startswith("javascript:"):
        # external link, do not check
        pass
    elif arg.find("#") == 0:
        # anchor, skip
        pass
    else:
        fullPath = os.path.dirname(FILENAME) + "/" + arg
        if not os.path.exists(fullPath):
            report_error("relative file %s doesn't exists" % fullPath)

def templatePathToPythonPath(templatePath):

    return string.join(templatePath.replace("/views/", "/controllers/").split("/")[:-1], "/") + ".py"

def eq(got, expected):
    if got != expected:
        print "got:'%s' != expected:'%s'" % (got, expected)
        return False
    return True

assert eq(templatePathToPythonPath(
    "web2py/applications/ad/views/default/details.html"),
    "web2py/applications/ad/controllers/default.py")
assert eq(templatePathToPythonPath(
    "web2py/applications/ad/views/default/index.html"),
    "web2py/applications/ad/controllers/default.py")
assert eq(templatePathToPythonPath(
    "web2py/applications/examples/views/ajax_examples/index.html"),
    "web2py/applications/examples/controllers/ajax_examples.py")

name_to_contents = {}
def get_file_contents(fileName):

    global name_to_contents
    if not name_to_contents.has_key(fileName):
        if os.path.exists(fileName):
            f = file(fileName)
            name_to_contents[fileName] = f.read()
            f.close()
        else:
            report_error("Cannot load %s" % fileName)
            name_to_contents[fileName] = ""
    return name_to_contents[fileName]

def check_url_exists(url):

    if FILENAME.find("appadmin.html") > 0:
        return

    if url.find(".") > 0:
        functionName = url.split(".")[-1]
    else:
        functionName = url

    # print "check_url_exists(%s) moduleName=%s" % (url, moduleName)
    pythonFilePath = templatePathToPythonPath(FILENAME)

    if get_file_contents(pythonFilePath).find("def " + functionName + "()") < 0:
        report_error("cannot find %s in %s" % (functionName, pythonFilePath))

def check_url2_exists(moduleName, functionName):

    if moduleName == "static":
        return

    # print "check_url_exists(%s) moduleName=%s" % (url, moduleName)
    pythonFilePath = string.join(FILENAME.replace("/views/", "/controllers/").split("/")[:-1], "/") + "/" + moduleName + ".py"

    if get_file_contents(pythonFilePath).find("def " + functionName + "()") < 0:
        report_error("cannot find %s in %s" % (functionName, pythonFilePath))

def scan_file(path):

    global FILENAME
    global FNR

    FILENAME = path
    FNR = 0

    f = file(path)

    while 1:
        FNR += 1
        line = f.readline()
        if not line:
            break

        m = RE_SRC.search(line)
        if m:
            check_src_exists(m.group(1))

        m = RE_HREF.search(line)
        if m:
            check_href_exists(m.group(1))

        m = RE_URL2.search(line)
        if m:
            check_url2_exists(m.group(1), m.group(2))
        else:
            m = RE_URL.search(line)
            if m:
                check_url_exists(m.group(1))

    f.close()

def test_html(directory):

    for a in os.listdir(directory):
        if a == "epydoc":
            continue
        p = directory + "/" + a
        if os.path.isdir(p):
            test_html(p)
        if a.endswith(".html"):
            scan_file(p)

test_html("web2py/applications")

# grep pl_PL.UTF-8 /usr/share/i18n/SUPPORTED > /var/lib/locales/supported.d/local
# dpkg-reconfigure locales
   Generating locales...
     pl_PL.UTF-8... done
   Generation complete.

Above example show how to add pl_PL.UTF-8 locale.

How to avoid that situation in a future? Should I rely on my memory? Is it possible for a human to track so many small rules manually? My intuition tells me that enforcement of those small ruleset should be automated.

GIT allows you to specify so called “commit hooks” that can validate many stages of GIT workflow. I’ll use simplest local verification of commit message, first the rule in plain text:

If you are changing moduleB you should notify developerC about this change

The implementation (.git/hooks/commit-msg):

#!/bin/sh

if git st -s | grep -q moduleB; then
    grep -q '#reviewthis:.*developerC@xyz.com' $1 || {
        echo "Add #reviewthis.*developerC@xyz.com"
        echo "Commit aborted"
        false
    }
else
    true
fi

If you make a change in moduleB and developerC is not notified commit operation is aborted. You cannot forget (or else your commit will fail). If you want to make that rule repository-wide (in order to block everyone from skipping this rule) you should use different hook: pre-receive on a server.

GIT hooks are very powerful validation / automation mechanism that is making our work more efficient.

… that’s the question.

Author: unknown.

On the other hand general problem with DFD (and any diagramming technique) is scalability. If you want to express many flows using DFD you have to make DFD hierarchical, thus overall picture may be hidden here. If you place too many (>9 items) details on one diagram you are lost again.

There’s simple and very powerful, but forgotten technique called “CRUD tables”. What’s “CRUD”? It’s a table (can be implemented by a spreadsheet) that for every process in a system defines it’s interactions with main entities: Create, Read, Update, Delete. CRUD table was used mainly during database design, but I think this technique can be used for every data flow in a system (including external actors and entities).

I’ll show you how you can utilize CRUD table to make initial design and get software complexity estimation.

First of all, a specification:

Pretty short, but let’s start design!

We can see how basic processes interacts with internal data (Product, Order, Item) and external entities (Payment, ERP Order). Let’s check first rule:

We can see rule is not valid for few processes, so let’s add some Entities to fulfill this rule:

As you can see we added actor (Customer) to be a source (trigger) for some processes.

Now we can add estimation part. First: for every entity we can add complexity (we know that one entities can be more complicated than another), complexity of processes will result from used entities complexity:

Formula used for computation of “complexity” column:

=counta(C3)*$C$2+counta(D3)*D$2+counta(E3)*E$2+counta(F3)*F$2+counta(G3)*G$2+counta(H3)*H$2

So we know now that more complicated process will be sending order to an ERP system. If we added proper weights to operations (CRUD) and measure single activity we could estimate full effort needed to implement specified system.

Above CRUD translates into the following DFD:

I’d like to hear about your experience with CRUD modeling.

First of all you need root access on a machine to install startup script. Default startup scripts are placed in /etc/init.d directory and special symbolic link must be created under /etc/rc5.d (at least on Debian-based systems).

First let’s create startup script for your “daemon”-like program:

/etc/init.d/myprogram:
----------------------
#! /bin/sh
PATH=/sbin:/bin
case "$1" in
  start)
    su -c /home/darek/bin/myprogram.sh darek &
    ;;
  restart|reload|force-reload)
    killall myprogram.sh
    su -c /home/darek/bin/myprogram.sh darek &
    ;;
  stop)
    killall myprogram.sh
    ;;
  *)
    echo "Usage: $0 start|stop" >&2
    exit 3
    ;;
esac

Note that “darek” is the user id that will be used for starting up the script.

Then: ensure it’s executable and create symbolic link for default runlevel (5 on Debian):

chmod +x /etc/init.d/myprogram
cd /etc/rc5.d
ln -s ../init.d/myprogram S80myprogram

“S” means “start”, “80″ allows to ordered execution. Pretty simple!

SYSV init scripts are harded to setup than BSD-style, but are more flexible and easier to manage by installation tools (dpkg under Debian).

• Someone has not predicted errant situation during specification phase
• OR: your waterfall (yes, it’s a linear analysis-project-implementation-test flow!) process is basically not working now

I believe without formal methods you are not able to predict all possible specification inconsistencies before implementation. Even very accurate review process might leave some holes in specification that lead to obvious bugs as a result.

Then usually a customer is forced to:

• accept extra work that needs to be done to remove a bug to be paid separately (so called “change request”, usually pretty expensive at the end of a process)
• OR: accept “feature” as is

Why? Because he signed specification! He/she wanted to make fixed-price product so contract must be created. Contract is not perfect, so responsibility for validating it is on contracting side, not the contractor. On the other hand we (the software shop side) like “THE SOFTWARE IS SUPPLIED WITH NO WARRANTY” (note the usual caps) that keeps our responsibility far away.

So: how often your software process introduces such “(cre/fe)atures”?

First, you have to include sources from fresh system version, below is example taken from Ubuntu, I selected natty (/etc/apt/sources.list):

deb-src http://pl.archive.ubuntu.com/ubuntu/ natty main restricted

Then you should refresh package list:

sudo apt-get update

and install dependencies:

sudo apt-get build-dep ccache

and finally build the new version of a package (note that no root account is needed for that step):

apt-get -b source ccache

As a result there’s a *.deb package, install it:

sudo dpkg -i ccache*.deb

That’s all!

I’m tracking current project state using automated test suite that is executed 24/7. It gives information about stability by randomly exploring project state space. Test launches are based on fresh builds from auto-build system connected to master branch for this project.

Recently I hit few times typical stability regression scenario: N+1th commit that should not have unexpected side effects caused crash in auto-testing suite just after 2 minutes of random testing thus blocking full test suite to be executed. OK, we finally had feedback (from auto-test), but let’s compute the (bug .. detection) delay here:

• auto-build phase: 30 minutes .. 1 hour (we build few branches in cycle, sometimes build can wait long time to be performed especially if build cache has to be refilled for some reason)
• test queue wait phase: 30 minutes (pending tests should be finished before loading new version)
• review of testing results (currently manual): ~1 hour (we send automatic reports, but only twice a day)

In worst scenario we may notice the problem (regression) next day! Too long time in  my opinion – broken master branch state may block other developers thus slowing down their work. There must be better organization that will allow to react faster on such problems:

1. Ask developers to launch random test suite locally before every commit. 5 minute automatic test should eliminate obvious crashes from publishing on shared master branch
2. Auto-Notify all developers about every failed test: still need to wait for build/test queue, messages may become ignored after long time
3. Employ automatic gate keeper-like mechanism that will promote (merge) commit after some sanity tests are passed

Looks like 1. is more efficient and easiest to implement option. 2. is too “spammy” and 3. is hardest to setup (probably in next project with language with ready support for “commit promotion”).

2. will look interesting if applied to latest submitters only. Maybe I’ll give it a try.

Those assertions typically will show filename/line number/message information and abort program / raise an exception if the condition is not met. Runtime environment then can collect current stacktrace to give developer more information on failed expectation.

One can disable assertions entirely (C++, Java) or select subset of assertions (Eiffel) for production releases. Resulting code will be faster, but failed expectations will not be verified by software – problem reports may be harder to interpret.

On the other hand if an assertion is too strict (the assumption may be invalid) it may abort program giving user negative impression about software stability.

What to do then? How can we keep problem-diagnosing power of enabled assertions and prevent minor of invalid failed assertions from aborting whole program?

The answer is: weak assertions (assertion without abort).

OK, you may ask, if we are not calling abort() then we can loose details about failed assertion. Nobody will send a bug report if program is not crashing.

But why do we need to rely on end user? Let’s send failed assertion reports automatically under-the-covers and leave system running!

I’m attaching solution below for Q_ASSERT (C++, QT), but you should get the idea:

void qt_assert_x(const char *where, const char *what, const char *file, int line) {
    char buf[BUFSIZE];
    snprintf(buf, BUFSIZE, "%s:%d: Q_ASSERT(%s, %s) FAILED", file, line, where, what);
    print_stacktrace_to_logs(buf);
    send_stacktrace_over_network(buf);
    if (getenv("ASSERT_FATAL")) {
        abort();
    }
}

If you place this code in LD_PRELOAD-ed library you will overload symbol from QT library and catch any reference to qt_assert_x() symbol. Supplied function will save important information into log file, send it using network connection (of possible) then return. I described in this post how can we collect such crash reports on a server and this post will tell you about implementation of stacktraces for C++.

Optionally you can ask for abort() call (typical Q_ASSERT behavior) using some kind of configuration (environment variable in my case). It may be useful in development environment to die early is something goes wrong to have better diagnostics.

Using custom error handlers you can have both benefits:

• “robust” program that will not die after minor expectation is not met (end user will not care)
• automatic diagnostics about failed expectation

Glossary:

• Customer: calendar owner
• Client: wants to book an appointment with customer
• Time slot: period of time that can be selected for appointment

MVP Use case list below (will define implementation order and priorities):

• Show customer’s calendar on a website with available slots
  • WHY: Proves that we can read existing event source
• Allow to book an appointment for selected slot
  • WHY: Proves that we can write existing event source
• Allow customer to register new calendar
  • WHY: Allow for external tests for potential customers
  • Allows to set timezone, day of week range, hours range

Optional use case list:

• SMS book confirmation
  • WHY: prevent abuse
  • Phone number is required from client
  • SMS code should be entered on confirmation screen to finish booking a visit
• SMS notifications
  • WHY: minimize no-shows
  • One day before a visit notification SMS is sent
• Vacation management by Customer
  • WHY: block booking for selected days
  • allow to specify date range for “unavailable” days
• Automatic SMS after moving a reservation
  • WHY: easy change schedule
  • compares local cache with current calendar state
  • sends SMS to client and e-mail to customer
• Custom SMS messages (with proper character limit check)
  • WHY: more details (address) in a message
• Client cancels appointment
  • WHY: minimize no-shows
  • Maybe as easy as sending SMS to some address with confirmation?

To be continued …

Above sample is not bad system responded in <1s for >88%. It looks like no optimisations needed here. Let’s see different case:

Ops! We had some performance problems 2010-03 .. 2010-07 (twitter.com, BTW). Looks like they were fixed in next months.

We are going to allow store and show day-by-day visualisations as well to give you feedback in shorter amount of time.

• In-place branches (I used to use them heavily)
• Faster (no Python libs loading during “cold” start)
• Can’t live without “git rebase -i” now

First you have to generate key pair:

$ mkdir -p ~/.gnupg
$ gpg --gen-key

Then see your new key ID and export it to public key storage:

$ gpg --list-keys # get KEY_ID from output
$ gpg --keyserver "hkp://subkeys.pgp.net" --send-key <KEY_ID>

On remote machine import the key and make it trusted (to avoid warnings during encryption):

$ gpg --keyserver "hkp://subkeys.pgp.net" --recv-keys <KEY_ID>
$ gpg --edit-key <KEY_ID>
> trust

Then you can used this key to encrypt files and delete original (if needed):

$ gpg -r <KEY_ID> --output <FILE>.gpg --encrypt <FILE>
$ rm <FILE>

And the decryption (on host where private key is stored):

$ gpg -r <KEY_ID> --output <FILE> --decrypt <FILE>.gpg

• WebStack + mysqldb + custom template framework (long-term ERP project for small company)
• Django (online shop)
• CherryPy + Cheetah (online PDF design)
• Zope (financial)
• WebStack + SimpleTAL (online PDF design)

All frameworks have its strengths and weakness. For new project that will handle appointments using existing calendar I decided to give web2py a try, rationale:

• all stuff included on board, no manual integration of 3rd party libraries
• stable API
• small and elegant
• integrates with GAE (with subset of DB layer)
• template selection separated from controller (easier unit testing)
• easy template syntax (reuses Python code embedded into markup language)

After first phase of product I’ll report if my expectations above were correct and what kind of problems were located (if any).

I was asked recently about integration with hosted Microsoft Exchange calendar. Customer (also medical business) is using currently iPhone to record visits and would like to use existing contacts and calendar events and allow his customers to book online.

Our current persistence mechanism is local SQL database. In order to integrate with external contact / event source we have the following options:

• keep local SQL storage and make periodical import/export of events/contacts to Microsoft Exchange
• replace local SQL storage with remote-only storage thus import/export will not be needed

First option drawback is possible de-synchronisation / higher probability of conflicts. The benefit is that local storage is more reliable than rely only on remote system availability. Also some customers doesn’t have existing calendars we can connect to. Then setup is much simpler (just signup to our online service).

The latter gives better guarantees of eliminating conflicts, but may be slower / less reliable (when remote system is unavailable for some reason).

We started few weeks ago attempt to integrate with Google Calendar – the interface is pretty well documented. Current option that will probably win is to rely only on remote storage with some data cached locally to make things go faster. Maybe the same approach will be used for MS Exchange (probably CalDAV as transport option).

0 8 * * *     df -h | awk '/^\// && $5 > 95 { print "missing space", $0 }'

and ensure e-mail to this user is forwarded to you then you will see e-mail when occupied space is higher than 95% on any disk.

Pretty simple.

In order to track the problem I redirected syslog (busybox-based, BTW) via network and added local automatic monitoring service that will show me when a part of my test installation go down.

The script is really simple and uses GNOME notification tool called notify-send.

#!/bin/sh
if ! ping -q -c 1 google.com > /dev/null
then
    # no network present
    exit
fi
for machine
do
    F=/tmp/$machine
    F2=/tmp/$machine.previous

    if ssh $machine 'echo OK' >$F 2>&1
    then
        rm -f $F $F2
    else
        if test -f $F2
        then
            notify-send "$machine is not present: `cat $F`"
        fi
        mv $F $F2
    fi
done

Details:

• I’m checking if network is available in general (google ping)
• List of SSH machines given on command line
• I assume SSH keys are setup – no password prompt
• Check state is held in /tmp/ directory

Script is started periodically from crontab:

* 9-17 * * 1-5    srv-monitor-ssh alfa beta delta zeus zeus2

and reports failure on second failed check.

• invalid name (wrong artifact number)
• invalid comment inside (also based on wrong artifact number)

it was a mistake and programmer wanted to preserve changes done on branch, but using different name.

The solution I proposed was to:

1. clone existing branch under different name
2. “amend” last commit on this new branch (to fix comment)
3. push new branch into correct location
4. drop old branch

Sequence of GIT commands was:

$ git branch corect-name origin/invalid-name
(We saved desired SHA with new name)
$ git checkout corect-name
$ git commit --amend
(Fix comment of last commit in editor)
$ git push origin corect-name
(Fixed branch published on our default remote)
$ git push origin :invalid-name
(Weird GIT syntax for deleting remote branches)

For GNOME command line interface to internal message is notify send. I’m adding call inside my ~/.procmailrc (after filtering out spam e-mail of course):

:0hc
| grep "^From:\|^Subject:" > /tmp/mail.notify-send; DISPLAY=:0.0 notify-send "`cat /tmp/mail.notify-send`"

The result looks like this:

• Automatic crash reports collected on central server
• Automatic random test runners connected to always-running (24/7) devices to catch crashes

First channel collects all crashes (from human and automated tests), second channel is performed fully automatically. Second channel allows to measure MMTF (mean time between failures) and analyse changes in time, probably helping with estimating current software quality state.

Second testing channel requires automatic test driver to inject random UI events (key presses, remote in my case). I used QT message queue for that purpose:

void TestScript::sendKey(int keyCode) {
 QWSServer::sendKeyEvent(0, keyCode, Qt::NoModifier, 1, false);
 msleep(PRESS_DELAY_MS);
 QWSServer::sendKeyEvent(0, keyCode, Qt::NoModifier, 0, false);
 msleep(WAIT_DELAY_MS);
}

Because QWSServer::sendKeyEvent() must be called in the same process like main application I decided to implement named pipe that will accept key names and render sequence of key events, pseudo-code below:

    while (true) {
        FILE* f = fopen("/tmp/events.pipe", "rt");
        (...)
        while (getline(&buf, &size, f) >= 0) {
            QString line(buf);
            (...)
            if (mapNameToKey.contains(line)) {
                sendKey(mapNameToKey.value(line));
                QCoreApplication::processEvents();
            }
            (...)
        }
        fclose(f);
    }

And usage:

echo "RCU_RIGHT" > /tmp/events.pipe
echo "RCU_OK" > /tmp/events.pipe

That’s way I’m able to remotely control application (need just SSH access to the device). And the last (easiest) part is to generate random sequence of events – that can be done by any scripting langugae.

Happy automation!