Dariusz on Software

Methods and Tools

About This Site

Software development stuff

Archive

Entries tagged "django".

Django 1.1 released
Thu, 30 Jul 2009 07:40:03 +0000

New version (1.1) of very popular web framework Django has been released. New version includes ORM improvements, better testing performance, conditional view processing and URL namespaces.

ORM Improvements

One of SQLObject benefits over Django ORM was in my opinion  possibility to make aggregations without issuing raw SQL-s. This feature is now present in Django allowing to make more efficient reports implementation.

Model improvements

A table can be marked now as "unmanaged": syncdb/reset will not touch such table. Also: proxy models and deferred fields now are present.

Testing improvements

Big optiomalisation for unit testing fans like me: tests are run in one transaction, so performance is much better than before. I don't use Django unit test infrastructure (selected nosetests for this task), but it's pleased to hear about this improvement. Hope it will make unit testing more popular among Django programmers.

django-logo-negative

Tags: django, python.
Migrate Database Schema in Django
Wed, 07 Oct 2009 19:49:59 +0000
When your application grows your database must be extended together with application needs. Schema migrations could be a pain if done improperly. Let's review some methods to do schema migrations in your database.

Manual schema migrations, single database instance

This approach assumes you are using database administration tool to manually change your database schema to reflect application. Typically there's one single "master" database that has "official" schema and is used for tests and (possibly cloned version) for production. It's the way most novice programmers attack the problem. This approach is simple and strighforward, but does't have more benefits. Problems that may occure:
  • It's problematic to develop in paralell on branches: you need to change schema but other developers will see your changes and may break their code
  • schema migration is chaotic - it's very easy to forget add some field on production database

Migrations by series of SQL scripts, manual apply

Another, more advanced method, is to use series of SQL files that will ALTER database and optionally UPDATE some fields to reflect migration. You can order those scripts by prepending date and make all developers manually path their databases. Benefits:
  • Arbitrary schema modifications can be implementaed this way
Some problems to notice:
  • It's still very easy to forget applying some patches (or applying them in incorrect order) making your env broken
  • Additional work is needed to prepare scripts

Migrations by series of SQL scripts, applied automatically

A variation of above method. SQL migrations scripts are recorded (by name) after apply in special database table. They are applied automatically (system lists special upgrade/*.sql directory) and already applied patches are silently ignored. Benefits:
  • Automatic retest of whole sequence of patches (load old database dump and run upgrade)
  • Any compicated schema migrations could be implemented this way
  • Automatic order of execution
  • Exclude "double execution" problem (patch names recorded)
This solution is used by us and works perfectly for 6 years of continuous development (400 patches applied so far!).

Automatic schema migrations for ORM

ORM (Object Relational Mappers) have few possibilities to detect current database schema and compare it with existing model. Then migrations can be applied automatically on database to make it work with application. Examples: Using Django Evolution is very simple: 
./manage.py evolve --execute --hint --noinput
Similar for Deseb: 
./manage.py evolvedb --noinput

Which approach should I choose?

I suggest to start with Django Evolution (or others ORM equivalent) then if migrations become more advanced switch to registered, applied automatically SQL scripts. This way you will have speed at the initialisation of a project and later - more confidence with manually created SQL patches. 1
Tags: django, python.
How To Migrate Django To Different Database Backend
Sat, 24 Apr 2010 21:41:57 +0000

Changing database location is simple - just launch dump on source database server, import it into destination database, redirect domain and voila! You can use this method to migrate your database into newer database engine version. But what can you do if you realize whole backend must be changed (i.e. from MySQL to PostgreSQL)?

Migrating SQL dump to different database dialect is not very easy (column types / dates formats as first examples come to mind). But you don't have to operate on SQL dumps. The simple answer here is: "dumpdata".

Django uses special manage.py script to manage typical operations like: initialisation of database, preloading data, dropping database etc. The command:

manage.py dumpdata appname

prints on stdout all data contained in appname in universal Json format. Then you can load dump just created by using:

manage.py sqlreset gabinet | psql ...
manage.py loaddata filename.json

Database state must be reset before import. That's why sqlreset is used. sqlreset alone prints DROP DATABASE statements on stdout allows to purge database from tables (if passed to SQL execution tool).

Additionally you can gzip JSON data created to make migration (much) faster:

manage.py dumpdata appname | gzip -c | ssh destinationserv 'cat > data.json.gz'
(login to destinationserv ...)
manage.py sqlreset appname | psql ...
gzip -dc data.json.gz | manage.py loaddata -

Happy migrating!

Tags: django, python.

RSS feed

Tags

Created by Chronicle v3.5