chroot plone buildouts

the problem

Anyone who uses Debian or Ubuntu knows the golden rule: use apt-get. Apt is a best-of-breed software version and dependency management tool. And, on first sight, easy_install looks like it's sister in the Python department. Sadly, apt-get and easy_install turn out to be part of the problem: both create global-scope system-wide python installs. Also, they don't get along very well.

A polluted global PYTHONPATH makes it all but impossible to control what code gets called in your specific python project. We got ourselves a nice open source variant of DLL hell here. Anything might be depended upon by anything. Nobody keeps track.

If any one of the following conditions is true, you're in trouble:

1. You're running a Debian/Ubuntu system with lots of "python-*" packages installed with apt-get
2. You're running an up-to-date Ubuntu system that uses python2.5
3. You have installed code from the 'net using easy_install. Then you find out there's no easy_uninstall

On my workstations, all three hold true. So I had to find a way to get rid of all my mission-critical legacy python2.3 code (1), and get rid of my cutting-edge Ubuntu python2.5 environment (2), and prevent new development projects interfering (3).

On top of that, I needed to do all of this in a way that would easily synchronize development projects (and environments) between my laptop, my desktop, my team mates and our hosted deployments.

the solution

What we need, is per-project local-scope package versioning and dependency management. There's several tools available for exactly that purpose: virtualenv, zc.buildout. But. You still need a clean global python environment to use them properly. That's where chroot comes to the rescue.

clean chroot

Chroot basically creates a one-way mirror: you can look in, but you can't look outward from the inside. You can leave your legacy environment intact, in the "outside" "real" system. As soon as you enter the chroot, you find yourself in the clean, uncontaminated, "new" chroot filesystem, with a blissfully empty PYTHONPATH.

Chroot isolates your development environment from outside interference. Setting it up is quite easy. Here's what it looks like on Debian/Ubuntu:

# first, create a chroot environment
apt-get install debootstrap schroot dchroot
mkdir /opt/chroot
debootstrap etch /opt/chroot/etch http://ftp.nl.debian.org/debian

# configure schroot 
cat >> /etc/schroot/schroot.conf << EOF
[etch]
description=Debian etch
location=/opt/chroot/etch
users=yourusername
priority=1
aliases=default
EOF

# as root, switch to chroot 
chroot /opt/chroot/etch
# create user account
adduser yourusername
# in chroot, as root, install base packages for plone development
apt-get install gcc libc-dev libc6-dev python2.4-dev 
apt-get install python2.4-imaging python2.4-setuptools
# install some extra development tools
apt-get install openssh-client git-core jed
# now install zc.buildout
easy_install ZopeSkel
# done!
exit

Now you can enter your chroot, as a non-privileged user, by doing:

dchroot

isolated zc.buildout project containers

We installed zc.buildout using easy_install, but from now on you'll work the other way round: you use buildout to manage your easy_install eggs for you.

In fact, every plone2.5_buildout or plone3_buildout contains a complete zope+plone installation of it's own, plus any eggs you depend upon for the project you're using this buildout for.

So, we're using chroot to shield our development projects from legacy interference; we use plone buildouts to shield our development projects from eachother.

The next steps you need to take are documented excellently in the buildout tutorial at Plone.org. Please follow those instructions.

You'll generally want to version-track your innermost development product, see below. If you're also verion-tracking the containing buildout configuration, make sure to exclude lots of stuff that is auto-generated or downloaded from the 'net. I'm using the following .gitignore in my plone*_buildouts:

bin
eggs
develop-eggs
downloads
var
parts
*~
*py?
.installed.cfg

buildout development eggs

Contained within the buildout within the chroot, we finally arrive at the location where we can actually do code development. You use buildout to initialize a (new-style) egg in ./src/ or an (old-style) Product in ./products/. Again, for instructions see the tutorial elsewhere.

Make sure not to confuse the different types of buildout you're using:

plone2.5_buildout or plone3_buildout

these are complete plone+zope runtime environments that contain the product you're developing, and everything your product depends upon.

plone2.5_theme, plone3_theme, plone3_portlet, plone

these are code templates for initializing the actual product you'll be developing.

finally

The setup described above combines chroot, buildout and eggs in a very specific way, which addresses the shortcomings of easy_install while retaining it's benefits.

If all of this looks like an awful lot of work to you: it isn't. In less than an hour, you too can have a clean, fully controlled Plone development environment. You'll be amazed at the amount of productivity and fun you'll experience by cutting over to the new, agile development practices that are typical of Zope3 and Plone3.