Making a Django installation easier to move

Or, making settings.py less sensitive to project directory path

I typically develop a Django project on my own machine, under a sub-directory of my home director named for the client.  In deployment I may need to have the project within the home directory of a user named for the client, or for the website, or whatever compromise username was available from a shared hosting provider.  There may also be restrictions on the sub-directory path that can vary by provider.  And there may be a staging server with differing path restrictions.

The upshot is that those absolute paths that we need to put in settings.py must often differ across the several installs.

Yet most of settings.py is common across the installs, and is worthy of being kept under revision control.

I've seen and worked with or implemented a variety of schemes:

  1. Importing a module early in settings.py which has only an indicator variable, used by conditionals later in settings.py to select among various settings.  The additional module is not under revision control, or is one of several selected using a symbolic link which is not under revision control, making it easy to keep it distinct on the different sites, yet easy to regenerate should it be accidentally deleted.
  2. Have settings.py attempt to recognize where it is running be examining, for example, the host name or the USER environment variable, and set an indicator variable itself, for use by conditionals lower down.
  3. Have settings.py, near the end. do "import * from local_settings" inside a try-except-pass block.  This allows replacing any global in settings simply by defining its replacement in local_settings.py .  I especially enjoy this for keeping things like database passwords,  the site key, any web service credentials, out of a potentially public DVCS.  local_settings.py is, obviously, not kept in revision control, or at least not in the same, public,  revision control system.

The pinax folks take #3 to the next level by, within local_settings.py, obtaining access to the still not finished being imported settings module, letting them edit existing items, rather than replacing them.  Using this, for example, you could set the database password and, perhaps, host (if you're allowing remote access to the database) without duplicating the engine, database name, etc., in local_settings.py .  (I haven't made up my mind as to whether this access to the partially imported settings module if fragile. I suppose that its module level namespace must exist, but might import change such that you can't find it the way pinax does?)

Versions #1 and #2 tend to make settings.py messy, having to have a bunch of conditionals and several versions of some settings.

Version #3 is almost necessary for some things, but using it to react to a change in path to project feels wrong, almost like a de-normalization: If you move or install a copy of the project at a different path, you have to correctly change ALL of the corresponding settings.

A half way measure would be to define a variable containing the project path, and use it in the definition of all the path dependent variables.  This leads to an approach like #1 or #2 to allow it to vary by deployment.

There is, however, a better way (and I can't claim to have invented this, but it still doesn't seem common in the community).  The automatically defined variable "__file__" is the key, providing as it does, a path to the settings.py file itself.  It is unlikely to be absolute, but os.path.abspath() will fix that, and then os.path.dirname() will give you the directory containing the settings.py file.  Pre Django 1.4, this would have been the most likely choice as a base for your path dependent settings.  But starting in 1.4 there is, by default, an extra layer of directory, leaving manage.py one level up from settings.py.

You could still elect to put templates, the extra static from which to collect, and the served static and media folders at or below the directory containing settings.py , but I prefer to put these at or below the directory containing manage.py , so in 1.4 and beyond I would need an extra application of os.path.dirname().  But I'm actually also interested in the name of the directory containing settings.py , because it IS the project name, so I use os.path.split() and get both in one operation:

PROJECT_ROOT, PROJECT_NAME = os.path.split(os.path.dirname(os.path.abspath(__file__)))

or, for pre 1.4:

PROJECT_ROOT = os.path.dirname(os.path.abspath(__file__))

PROJECT_NAME = os.path.basename(PROJECT_ROOT)


(You can, of course, get the django version and choose which to do automatically.)

 

Now you can use os.path.join() to build up those path dependent settings, though if you might run it on Windows, you will also need to replace the backslashes with slashes.  But as a lazy guy who doesn't like to type much, nor read from among any more  boiler plate than I have to, I tend to define a little function ("_p" is short, but won't collide with anything that I know about):

def _p(*args): return os.path.join(PROJECT_ROOT, *args).replace('\\', '/')

(Since I haven't suffered under Windows recently, the replace part is untested.)

Then I can say, for example:

TEMPLATE_DIRS = (_p('templates'),)

MEDIA_ROOT = _p('htdocs', 'media', '')

(The final empty string causes MEDIA_ROOT to end with a '/'.  I could have just passed one string with two embedded slashes, but I figured I'd show the flexibility, should you happen to have one of the components, say 'htdocs' in a variable.  And yes, in today's "collectstatic" world, I like to put static, media, and things like robots.txt and favicon.ico in a sub directory.  I can cover it all with a single apache <Directory> directive.  And I can keep _p('static') to put in STATICFILES_DIRS, which is less confusing for designers who also regularly work on older versions of Django.)

I typically also do:

def _m(*args): return '.'.join((PROJECT_NAME,) + args)

allowing:

ROOT_URLCONF = _m('urls')

WSGI_APPLICATION = _m('wsgi', 'application') # again, I could have used just 'wsgi.application')

Judicious use makes the settings files completely independent of the project name.  That includes any apps you choose to install in the directory containing settings.py, rather than in the directory containing manage.py (which need no prefix).  So rather than editing what startproject provides as a settings.py, you could just replace it with a standard starting version that you keep around.  (You will want to check on it for new Django version, and even dot releases to see that you don't need something else.)

You can do similar tricks with the wsgi.py in the same directory as settings.py to make it independent of the project name.  You may also want to calculate PROJECT_ROOT there, so that you can make sure that it is on sys.path, simplifying your wsgi server configuration, though, note that the wsgi server still has to find this file, and the last path component is the project name, so it hasn't disappeared all together.  It also still needs to be in manage.py. (It seems unreasonable and fragile for manage.py to search sub-directories that are siblings for a file named settings.py .  The alternative is to assume that the directory containing manage.py is also named for the project, which always works pre 1.4, and, since startproject names it the same as the project, this works in 1.4, unless you rename the outer directory, see discussion below.)

While the outer directory, by default, is named the same as the project package (package in the sense of having an __init__.py file), in reality, nothing within Django actually cares what the name of this directory is, other than those path based settings, and we have handled that.  This is useful in that, at need, you run two copies of the same site in the same account, on the same box, simply by putting a copy of all the contents of the outer directory into another, differently named directory.  (This is why I typically make PROJECT_ROOT also be the root of my revision control tree: I can set up a copy with a single hg clone command, and maybe making a modified local_settings.py .)  I have done this when a problem is exhibited only on the production box, and I can't go sticking break points in the production server.  (Do beware of putting break points in site-packages code, unless you use a separate virtualenv.)

I have been sorely tempted to name all my 1.4 projects proj_rel or even my_site (I could see "project" colliding with an app), since the project package primarily serves as a way to disambiguate references to modules that get their names reused within apps, just in case sys.path gets set up funny, module names like urls, settings, and even wsgi and local_settings.   I can still have multiple projects on a box, because the directory containing manage.py and the project package can still be named whatever I want.  Then I could keep around a tar or zip file containing my starting configuration, rather than running startproject.