Deploying Django

The INTERBLOGZ are abuzz today with talks of shared hosting and web frameworks. Dreamhost started the bal, there was a response from DHH and James Bennett has his own thoughts on the matter.

We use Django at work because it makes the development and maintenance phases of development so fast, easy and pleasant, but I can tell you that the deployment of those applications is a pain in the ass. We need to use three different .htaccess files to make our applications work correctly and every time we make a modification to a Python file, we need to restart Apache (we know the hosting company we do business with well, and they agreed to let us have root access.) PHP may be a pain to work with, but at least deployment is a simple upload.

Without going into too much details, on the hosting server, clients have a folder with a bunch of subfolders in it, one of which is httpdocs/. This is the root of their web site. Because our skeleton project (see previous blog post) is called ‘wsf’, we need a way to redirect from http://foo.com/ to http://foo.com/wsf/. That’s what our first .htaccess file is for:

RewriteEngine on
RewriteBase /
RewriteRule ^$ wsf/

This file is located directly in httpdocs. Then, we need to tell Apache that wsf/ is a Django project, so we have a second .htaccess file in httpdocs/wsf:

SetHandler python-program
PythonHandler django.core.handlers.modpython
SetEnv DJANGO_SETTINGS_MODULE wsf.settings
PythonInterpreter client_name
PythonPath "['/var/www/vhosts/foo.com/httpdocs'] + sys.path"
PythonDebug Off

And finally, we have the media files in httpdocs/wsf/media. I begin by making a symlink in httpdocs (ln -s httpdocs/wsf/media .) and in that directory, I need to put the third .htaccess file:

SetHandler None

Django is an absolutely fantastic way to develop web sites and applications, but it’s a terrible way to deploy them. I think one of PHP’s greatest strengths (some might call it a downside) is that it’s extremely easy for somebody to write a script and get it working online. We get excited about Django, we get people interested, but when they ask the simple question “how do I get this online?”, our answer is “hmmm, that’s pretty complex…” It shouldn’t be.

A few weeks ago on the FLOSS Weekly podcast, Leo Laporte and Randal Schwartz were interviewing Avi Bryant of Seaside fame. Randal and Avi were both super excited about Smalltalk, Squeak and Seaside, so Leo downloaded Squeak during the show, booted it up and asked the question “How do I write ‘Hello world’?” And at this point, I swear I heard Avi and Randal say “fuck” in their heads. A similar silenced “fuck” came about when Leo asked how to install Seaside. I always found deployment in Smalltalk to be a hassle, and I guess just getting something to work easily isn’t its strong suit. Web frameworks should not be like this.

There needs to be more work done in that area with Django and other frameworks. We need to be able to just upload a project and have it work easily and painlessly. There should be no need to restart Apache when you make a modification to a Python file. You shouldn’t have to edit httpd.conf or .htaccess to get your application working (an initial setup in httpd.conf to enable Django support in general is acceptable.) Root access should not be a required to get a Django application up and running. (Technically, root access isn’t required, but you won’t find a lot of hosts that don’t mind you restarting their services.)

I vaguely remember trying the Symfony PHP framework, which was partly inspired by Rails. I do not recall having to make any special configuration in httpd.conf to get it working. Maybe Django and Rails should look in that direction for tips on how to get things working.

About these ads

15 thoughts on “Deploying Django

  1. Nice post co-worker :D

    I think our little *tricks* helps us a lot in the django deployment process, but this isn’t just the deployment. Another thing that is a pain in the ass with Django is the translations with gettext. Someone have to come with a simplier way to translate.

    Gettext is fine and does a great job, beside being forced to use english as a primary language. But having to type a gettext command and restart apache each time we make a change in the translation files is utterly unacceptable. There is way to much steps involved.

    Beside that.. <3 Django :)

  2. I find it curious that you consider restarting Apache to be such a big issue in deployment of Django. This, by the way, is not a Django-related matter at all, but a side-effect of using long-running processes which don’t refresh on every request like PHP or old-skool CGI-programs. (James Bennet explais this quite well in the post you linked.)

    How often do you make a deployment to a production server? Once a day? Once a week? To me even a manual restart — involving one command that takes half a second — is not a big deal. If you are _developing_ in an environment with Apache, you can (and should!) trim down the MaxRequestsPerChild to 1 and the server will reload itself on every request.

    But then again, I really think that when working with such a great development tools like Django or Rails, one should definitely follow the path all the way and learn to use automatic tools for deployment, too.

    With Capistrano, for example, you can write a small deployment script that will with *one command* 1) check out the new code to remote server from version control (Mercurial, Subversion, whatever you like), 2) update the database schema if necassary (and back up the old one), 3) reload your webserver(s) of choise, and finally, 4) run your test suite to test that everything works like it should. *And*, if something should go FUBAR (which never happens, right ;), you can rollback the whole thing with another command. I love automated deployment. I’ve never ever had the pleasure of fully controlled, 100% trouble-free, one-command deployments like I do with Django and Capistrano nowadays. And that’s only because I didn’t know better before!

    Lastly, poor support for things like mod_python or long runnin processes are not a fault of any framework; they are failure of a hosting company. (Would you complain of how terrible it is to get Symfony running if your host wouldn’t have mod_php set up and you’d have to set it up yourself?) There are plenty of great hosting companies that support Rails and Django, just pick your favourite! (Or better yet, just grab a Slicehost slice for $20/month and enjoy the total control of your server!)

    Happy deployments! :)

    PS. I’ve written an article descriping how to use Capistrano to deploy Django apps. It should get you started (fully working example script included!) if you’re interested in that.

  3. I have to agree I found myself wondering why they chose to use .htaccess files when the hosting company “agreed to let us have root access”. Sounds like they are hosting on a virtual – not that there’s anything wrong with that, but then root access is a given. If they have root access to a shared server I woud ike to know I wasn’t sharing the same server …

  4. Stu: Flickr use .htaccess files. The performance overhead involved is tiny compared to the real bottlenecks, and they have maintenance advantages (no need to restart the server to see your configuration changes).

  5. Ville: I will check Capistrano out. The reason we restart often is that clients often ask that we put their sites online as soon as possible, but as soon as it’s up, they notice things they want to change. When it’s template code, that’s all right, but if I add an URL, modify a view function or a form, I got to restart Apache. Sure it restarts in one second, but it’s still annoying. I wonder if it wouldn’t be possible for Django to automatically reload a list of files (specified in settings.py) if they changed…

    Stu: What would you recommend I use then? I would like to keep out of httpd.conf, I wouldn’t want to accidentally mess up the configuration.

    Steve: It’s not a virtual machine. It’s a Fedora Core server. I prefer not to use root unless I absolutely, positively need it. I wouldn’t want to fuck things up.

  6. As tony pointed out, unless you run a site like youtube and have a zillion hits per day, small performances costs that .htaccess have are trivial compared to how they help us to streamline the deployment process.

    I’m tired of people posting provocative, unconstructive and useless comments like this. The only thing you achieve is proving your blatant ignorance by not making any valid point.

    Why don’t you illuminate us with your great “professional” wisdom instead of just bashing the author with your insipid thoughts ?

    Throwing accusations is easy, proving them right is another story.

  7. Gnuvince: you’re certainly doing the right thing staying out of root unless you absolutely need it. However, the configuration elements you mention in your post seemed so fundamental to each site that they wouldn’t need tweaking on a daily basis. I wasn’t so much thinking about performance as centralizing control of the server.

    h3: sorry you’re tired. Nobody is throwing stones here (or if they are, learn to rise above them!). Get more rest ;-)

  8. There are some constraints to how much Python can be like PHP. Basically PHP runs with the model of a CGI script. That’s a great model to develop in. And it works okay when most of your application and language is implemented as C code in a shared .so file. I get the impression that it doesn’t scale up well; e.g., Symfony has some performance problems because it involves a lot of PHP library code that runs each request (from what I understand an accelerator like Zend’s is similar to what .pyc files do, which is important but not enough).

    That said, detecting changes is a bit hard, but a soft reset is all you need — you need to throw away the Python interpreter you’ve been using, but the Apache process can stay up. I think mod_wsgi has more options for that sort of thing, but I don’t know the details. Generally mod_wsgi seems to be more thoughtful about these issues. If you run with a system that proxies to another request (HTTP proxy, FastCGI, SCGI) then usually you can restart your long-running Python server without losing any requests.

    For handling libraries you need some kind of path isolation. This is fairly easy to figure out, but depends somewhat on your deployment situation, which is why it becomes confusing all over again. There’s too many choice branches, and any instructions anyone comes up with choose just one branch, and maybe not the one you want. E.g., I would never write up something about how to use mod_python, and so the pieces of the puzzle I might be able to address don’t get well integrated into other ideas about deployment.

  9. Pingback: Deploying Django with Mercurial, Fab and Nginx « Stream Hacker

  10. Pingback: Deploying Django with Mercurial, Fab and Nginx « streamhacker.com

  11. Pingback: throat cancer symptoms

  12. Pingback: WordPress Themes Django Camfrog | Camfrog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s