After recent OS upgrade one of my unit tests started to fail (to be precise it started to hang). Quickly check showed me that CGI process started by os.popen() hanged. The old source code:
f = os.popen("./cgi_script.cgi > /dev/null", "w")
As os.popen() is deprecated now (I know, it's a very old codebase that started with Python 1.5) I've moved to new subprocess module:
fNull = file("/dev/null", "w")
p = subprocess.Popen("./cgi_script.cgi", shell=False, bufsize=1024, stdin = subprocess.PIPE, stdout = fNull)
fw = p.stdin
As you can see it's more verbose now but I've eliminated shell (slightly faster operation).
Some notes found during migration:
- without "del p" process may be not terminated causing problems with DB state (CGI proces updates database and test checks this state later)
- I/O configuration is more flexible than os.popen() – you can make pipes more easily
Automated unit tests are hard to write. Software architecture must be designed carefully to allow unit testing. You have to spend time to write tests as well and it's not easy to write good tests. It's easy to make big mess that is hard to maintain after few weeks.
On the other hand automated integration tests are hard to maintain and are fragile. You can "write" them pretty easy in a record-and-replay tool, but later they show their real cost during maintenance.
But there's an answer for problems mentioned above. Do you know Eiffel language? The language has special built-in constructs that support executable contract specification. It's called Design By Contract (DBC). DBC is easier to write and to maintain because no special test cases need to be specified, just conditions for method parameters, expected results and state invariant that must be preserved. How DBC can substitute old-fashioned tests? Read on!
Web2Py is a full-stack Python web framework that can be compared to Django, but is easier to learn due to convention-over-explicit-statement preference. In this article I'll check how static verification techniques developed by me for many different environments (JSP, Django templates, TAL, …) can be applied for web2py environment.
Static verification means locating simple bugs without running application thus very high (>95%) testing coverage (and high related cost) is not required. Instead with trying to cover by tests every possible screen/workflow/code line/… we can scan all codebase and search for some constraints. Most of them (based on my experience) are static – do not depend on runtime data thus can be effectively checked without running an application.
I've been using many different Python Web Frameworks so far:
All frameworks have its strengths and weakness. For new project that will handle appointments using existing calendar I decided to give web2py a try, rationale:
- all stuff included on board, no manual integration of 3rd party libraries
- stable API
- small and elegant
- integrates with GAE (with subset of DB layer)
- template selection separated from controller (easier unit testing)
- easy template syntax (reuses Python code embedded into markup language)
After first phase of product I'll report if my expectations above were correct and what kind of problems were located (if any).
Changing database location is simple – just launch dump on source database server, import it into destination database, redirect domain and voila! You can use this method to migrate your database into newer database engine version. But what can you do if you realize whole backend must be changed (i.e. from MySQL to PostgreSQL)?
Migrating SQL dump to different database dialect is not very easy (column types / dates formats as first examples come to mind). But you don't have to operate on SQL dumps. The simple answer here is: "dumpdata".