10.09.05
Django Unit Testing
Looking through the unit tests for the Django project, I have been able to extract a relatively simple way of creating tests for my django models that do not effect production data or need to be cleaned up after each run. For my unit tests, I leverage sqlite and use an in-memory database that is thrown away after the tests complete.
First, I created a ‘tests’ module in my application with a new settings file that sets up the database.
DATABASE_ENGINE = 'sqlite3' DATABASE_NAME = ':memory:'
Then at the beginning of my unit test I need to prepare the database. The very first thing I needed to do was to replace the DJANGO_SETTINGS_MODULE module in the os.environ dictionary which will tell Django to load my test settings rather than my normal project settings.
Since the database is in memory and new on each run, the init and install commands need to be run. These commands are available through the django.core.management module. Below is the example startup code before my tests:
# Globals
SETTINGS_MODULE = 'apps.myapp.tests.settings'
TEST_MODELS = ('model_one', 'model_two')
# Settings Set First
import os
os.environ['DJANGO_SETTINGS_MODULE'] = SETTINGS_MODULE
# Install Models
from django.core import management, meta
management.init()
for model in TEST_MODELS:
management.install(meta.get_app(model))
# start tests
import unittest
class test_model_one(unittest.TestCase):
....
The most important thing is that the settings file gets set before importing anything from Django because Django will automatically load the ‘normal’ settings file the first chance it gets.
In order to use sqlite in python, you need the pysqlite project, which can be downloaded here:
http://initd.org/tracker/pysqlite
The following page includes the install directions which includes a simple test that can be run in the intepreter to make sure your install is correct:
http://initd.org/pub/software/pysqlite/doc/install-source.html
UPDATE:
I created a method that lets me now skip the creation of a settings file and reduces the amount of duplicate code. The trick is reloading the db module of Django.
def enable_inmemory_testing_db(models):
# Change Settings
from django.conf import settings
settings.DATABASE_NAME = ':memory:'
settings.DATABASE_ENGINE = 'sqlite3'
# Reload DB module
from django.core import db
reload(db)
# Install Models
from django.core import management, meta
management.init()
for model in models:
management.install(meta.get_app(model))
Also, a VERY nice side-effect that I didn’t realize at first is that I can now refactor my models and run my tests without having to drop and re-install my models using the django-admin script. Obviously, you still need to go thru the process for running your site but the TDD process of write a test, run it, modify, run it was slowed down a lot by the SQL admin stuff.
This was my biggest nit with Django and now it is eliminated.
akaihola Said:
April 10, 2006 at 2:11 am
For the magic-removal branch, some changes are needed:
def enable_inmemory_testing_db(models): # Change Settings from django.conf import settings settings.DATABASE_NAME = ':memory:' settings.DATABASE_ENGINE = 'sqlite3' # Reload DB module from django import db reload(db) # Install Models from django.core import management management.syncdb()It always prompts for creation of the superuser, which is annoying. Any ideas how to get rid of that easily? Rewrite syncdb?
akaihola Said:
April 10, 2006 at 2:14 am
Ok so the indentation isn’t preserved here and there’s no link to markup help.
Ian Maurer Said:
April 10, 2006 at 7:30 am
You can use the <pre> tags for formatting code.
As for the M-R branch, I haven’t had a chance to play around with it much. When I do, I will post an update here.
-ian
Ruben Said:
April 23, 2006 at 2:30 pm
akaihola,
to supress the “create a super user” question in magic-removal, I added 5 lines before the management.syncdb():
def enable_inmemory_testing_db(): # Change Settings from django.conf import settings settings.DATABASE_NAME = ':memory:' settings.DATABASE_ENGINE = 'sqlite3' # Reload DB module from django import db reload(db) # Install Models from django.core import management # disable the "create a super user" question from django.contrib.auth.management import create_superuser from django.contrib.auth import models as auth_app from django.db.models import signals from django.dispatch import dispatcher dispatcher.disconnect(create_superuser, sender=auth_app, signal=signals.post_syncdb) management.syncdb()I know this doesn’t look nice… Anyway, in my current magic-removal version, it works. I don’t expect it to have a bad side effect; it disables the “create_superuser” function ( which should be dispensable in unit test code).
Ruben
Ruben Said:
April 23, 2006 at 2:41 pm
next thing I aim to learn is using tags for well-formatted code …
but when I practise here, your blog is full ;-)
Ian Maurer Said:
April 23, 2006 at 2:53 pm
Excellent… I am going to try it out myself and do a separate post about this and the upcoming M-R branch.
Victor Ng Said:
May 4, 2006 at 1:36 am
Ruben, that last patch against MR won’t actually work all the time. I’m using Django’s trunk after the MR merge and there are lots of places in Django’s code where :
from django.db import connection
is used. So reloading db is not sufficient to make things work all the time as those modules that have imported ‘connection’ will have a reference to the old connection handle. I’ve submitted a patch to Django to fix this problem so hopefully we’ll see the fix merged into trunk soon.