dupdater.py is a Python utility for applying and removing an ordered sequence of updates to a Django project instance (codebase plus database). An update is typically used to evolve the database schema (preserving existing data) and is normally applied as part of a project codebase upgrade.

Updater is a deployment tool not a development tool — it's primary function is to assist rollout of project updates to live installations. You will probably already have tested your new development database schema by the time you write the update to roll it out.

Updater is not an automated schema evolution tool, there is no model or database introspection, you must write code to perform the application and removal project updates. Migrating a live databases is often a messy affair involving more than just the canonical “update database schema to reflect changes to the project models” (writing the SQL for schema changes, while usually easy for a developer, is a difficult problem to automate).

Updates are applied and removed in a fixed order based on an automatically assigned auto-incrementing update number (1…).

Updater is not installed as a Django application, it is a stand-alone utility.

Updater assists you apply, remove and manage updates:

  • Creates update skeleton from a boilerplate.

  • Records and tracks update application/removal.

  • Provides some useful helper objects and functions.

updater.py command

Usage: dupdater.py [OPTIONS] COMMAND PROJECT_DIR

Utility for applying and removing changes to a Django project. COMMAND can be
one of: create, history, list, update.

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -s SETTINGS, --settings=SETTINGS
                        project settings module name
  -P, --no-prompt       do not ask the user before updating
  -d DESCRIPTION, --description=DESCRIPTION
                        set update class doc string
  -u UPDATE_NUMBER, --update-number=UPDATE_NUMBER
                        target update number (1, 2, ...)
  -c, --current         history command only prints the current update number
  -n, --dry-run         show what would have been done
  -v, --verbose         increase verbosity

COMMAND

update Apply all outstanding (unapplied) updates. If the —update-number=UPDATE_NUMBER option is specified then apply or remove updates until UPDATE_NUMBER is the highest numbered applied update. An UPDATE_NUMBER of zero will cause all applied updates to be removed.
create Create a skeletal update module using the next available update number.
list List all update modules.
history List the updates_register table.

OPTIONS

Download

The current version is 0.6.0 and a distribution Zip file can be downloaded from http://hg.sharesource.org/dupdater/

The dupdater Mercurial repository is hosted by ShareSource. ShareSource is a Mercurial friendly website for hosting Open Source projects.

To browse the repo go to http://hg.sharesource.org/dupdater/.

Getting Started

First you need save a copy dupdater.py script, no other setup or configuration is required.

To create a project update use the create command and specify the root directory an existing Django project (the current directory in the following example):

$ cd myproject
$ python bin/dupdater.py create .

If this is the first project update an updates directory automatically will be created in the Django project directory.

$ cat updates/update_001.py
class Update_001(UpdateBase):
    """
    Put a description of the update here.

    """

    def apply(self):
        pass # Code to apply update goes here.

    def remove(self):
        pass # Code to remove update goes here.

Next step is to customize the update module updates/update_001.py with your favorite editor.

Finally apply the update, again you need to specify the root directory of your Django project:

$ python bin/dupdater.py update .
Create updates_register table? [y/n]: y
About to apply the following updates:

001: Put a description of the update here.

Continue? [y/n]: y

applying update: 1

Because this is the first time the update command has been run on this project the user has been prompted to create an updates_register table in the project's database.

The list command shows that update 1 is the only and the current update:

$ python bin/dupdater.py list .
--> 001: Put a description of the update here.

The first line of the update class doc string is the update description which can be edited at any time or set using the —description option when it is created.

Let's create and apply a second update:

$ python bin/dupdater.py --description='Drop gender field.' create .

$ cat updates/update_002.py
class Update_002(UpdateBase):
    """
    Drop gender field.

    """

    def apply(self):
        pass # Code to apply update goes here.

    def remove(self):
        pass # Code to remove update goes here.

$ python bin/dupdater.py update .
About to apply the following updates:

002: Drop gender field.

Continue? [y/n]: y

applying update: 2

$ python bin/dupdater.py list .
    001: Put a description of the update here.
--> 002: Drop gender field.

The listing now shows we have two updates and that update 2 is the current update i.e. all updates up to and including update 2 have been applied.

To remove update 2, reverting to update 1:

$ python bin/dupdater.py --update-number=1 update .
About to remove the following updates:

002: Drop gender field.

Continue? [y/n]: y

removing update: 2

$ python bin/dupdater.py list .
--> 001: Put a description of the update here.
    002: Drop gender field.

The listing now shows update 1 is the current update i.e. updates with numbers greater than 1 are not applied.

Update modules

Each update performed by a separate Python module residing in the <PROJECT_DIR>/updates directory. Update modules are created using the dupdater create command and are then user customized to perform update application and removal. Each new update is assigned a unique update number — the first update is number is 1, the second 2 and so on.

The update module is named like update_<number>.py and contains a same-named class which inherits from UpdateBase. For example the first update module (update_001.py) contains the class Update_001. An update class contains instance methods apply and remove which respectively apply and remove the update.

If the apply and remove methods are unable to complete successfully they must throw an exception and are responsible for performing any necessary rollbacks before exiting.

Update modules have access to the following dupdater.py global variables and helper functions:

Globals

PROJECT_DIR Django project directory.
SETTINGS Django project settings module.
OPTIONS Parsed command-line options OptionParser object.
UPDATES Project Updates object.

Helpers

exec_sql(sql) Execute SQL statement(s) against the project database.
exec_sql_file(filename) Execute file containing SQL statements against the project database.

Here's an example update module — the update drops renames the person table to persons; the remove method reverts the update by renaming the persons table back to person:

class Update_002(UpdateBase):
    """
    Rename person table.

    """

    def apply(self):
        exec_sql('ALTER TABLE person RENAME TO persons;')

    def remove(self):
        exec_sql('ALTER TABLE persons RENAME TO person;')
Tip
Use the —verbose and —dry-run options to help debug update modules (helper functions honor the —dry-run and —verbose options).

Files and Directories

dupdater.py The dupdater Python script.
<PROJECT_DIR>/updates/ Directory containing individual update modules. If the <PROJECT_DIR>/updates/ directory does not exist you will be prompted to create it when you first execute a command that accesses it.
<PROJECT_DIR>/updates/update_<number>.py Update module containing Update_<number> class. Skeleton update modules are created by the create command.

updates_register table

Every time a dupdater update command is run a new row is inserted to the updates_register table in the project database. dupdater.py uses the updates_register table to determine the current update number.

The inserted record's update_number is the number of highest numbered update that is currently installed (the so called current update number). A recorded update_number of zero means there are no currently applied updates.

If the updates_register table does not exist you will be prompted to create it when you first execute an update or history command.

Only one row is inserted per update command regardless of the number of updates applied.

If error occurs while processing an update command then the inserted record's update_number is the number of the most recent successfully applied update. A record is inserted in the updates_register table even if the update command failed to apply any updates.

Implementation Notes

All database interaction is via respective database command-line tools (sqlite3(1) for SQLite and psql(1) for PostgreSQL) — this is not as clean as the using Python Database API drivers but it does provide an easy way to execute arbitrary SQL script files.

The sqlite3(1) and psql(1) commands should be in the shell PATH (alternatively, on MS Windows, they can be in PROJECT_DIR/bin).

This version has been tested using Python 2.5.1, PostgreSQL 8.2, SQLite 3.4 and (to a lesser extent) MySQL 5.0 on Linux (Xubuntu 7.10). I've not tested PostgreSQL or MySQL on Windows (only SQLite).

Changelog

2008-04-13: Version 0.6.0 released
  • dupdater now hosted at ShareSource.

  • Changed name from updater to dupdater.

  • Added -c, —current history command option.

  • Moved UPDATE_NUMBER argument to —update-number=UPDATE_NUMBER option.

2008-03-05: Version 0.5.0 released
  • First public release.

References