Homebrew and Python on OS X

I'm working on a relatively large project in Python and I do most of my work on OS X. I've recently started working with more people and they also use Macs so we wanted to get a standard process for installing and maintaining our development environments on the Mac. Linux package managers really spoiled me and the Mac package managers such as MacPorts and Fink have never really worked quite as well. Other trouble I've had is with some of the Python libraries like PIL, ReportLap, and SciPy that do a lot of compiling of native code and depend on other libraries like FreeType2, libjpeg, etc. You need to build all of these and keep track of versions that are installed. This isn't a huge problem as these don't change as frequently, but my current client asked if the app we've developed can be installed on Suse 10.2 so we need to test with the same version of these libraries availible on that particular platform. Rather than recompiling everything everywhere, we can simplify and install the same versions locally. This has always been a giant pain though because you usually have one version of a library installed and that's it. Then my buddy Glen recently introduced me to Homebrew and I've been very happy since.

Homebrew has been described as the easiest and most flexible way to install the UNIX tools Apple didn't include with OS X. After using it for a day, I'd have to agree. I've used it to build a Python development environment and it looks like it will make a very maintainable environment on OS X. The whole point of this for me is to be able to maintain multiple versions of Python and any libraries that may be needed. I'm not going into any detail on the tools that are being set up here so if you need more info on Homebrew, virtualenv or virtualenvwrapper, check the links at the end of the post. Let's get started.

First install Homebrew. I followed the Homebrew installation instructions here: http://mxcl.github.com/homebrew/ This is an easy installation and sets the expectations for the rest of the Homebrew experience pretty high.

Then, using our Homebrew environemnt, install the prerequisites for our Python environment.

$ brew install readline sqlite gdbm

This should proceed without anything very exciting happening so we can now install Python:

$ brew install python --universal --framework

We do the universal install and the framework install to support 32-bit and 64-bit on OS X and the framework build to allow interaction with OS X libraries. I found this is also more reliable way to install multiple versions of Python. When you install with the framework option, Python is installed with the version appended to the name of the executables. This makes it easy to switch versions and have multiple versions installed so this is what we do here.

You should now have access to the Python 2.7 installation...almost. By default, Snow Leopard puts /usr/local/bin in the path, but it puts it at the end. I wanted my installations to take precedence so I edited /etc/paths and moved /usr/local/bin to the top of the file. The /etc/paths file is used by the OS X path_helper application that is called in /etc/profile which is the default profile that is called when you login. This sets the default for any shell to use /usr/local/bin first in the path. I also recommend learning about path_helper too because it is a key piece of OS X command line configuration. If you make this change and start a new shell your path should be updated. If you type in "python --version" you should see the latest 2.7 version installed now.

$ python --version
Python 2.7

Once this is done, we're ready to start setting up our Python development environments. For this, we're using virtualenv and pip. Pip is also a Homebrew recipe and we'll install that first.

$ brew install pip

This installs the pip and the distribute (easy_install) packages for Python. This lets us install more software for our Python environment using the Python Package Index. We can now install virtualenv and the excellent virtualenvwrapper scripts.

$ pip install virtualenv 
$ pip install virtualenvwrapper

This will now allow us to create a virtual environment and install the packages we need to work with. There a are few more steps to make things work smoothly. We need to link our virtualenv script that was installed to our /usr/local/bin directory. cd to the directory /usr/local/bin run the following command:

$ ln -s ../Cellar/python/2.7/Frameworks/Python.framework/Versions/2.7/bin/virtualenv virtualenv 

This puts the virtualenv script on our path and allows us to use virtualenvs for our Python environments. Then in our own .profile we need to add the following. Change the WORKON_HOME directory to the directory you want all your virtual environments to be created.

export WORKON_HOME=~/Dev/Envs
 source /usr/local/Cellar/python/2.7/Frameworks/Python.framework/Versions/2.7/bin/virtualenvwrapper.sh

At this point you should be able to start a new shell and start using your new Python development environment. If you install Python 3 or other Python distributions, you should not have to install virtualenv or virtualenv wrapper again. You can use virtualenv to create a new virtual environment with any installed version of Python by specifying the Python interpreter to use. To create your first virtual environment run the following command:

$ mkvirtualenv test 

This will create a new virtual environment called test in ~/Dev/Envs/test (or the directory you changed that too) and installs the Python interpreter, and the pip and distribute packages. You can now start installing software into your virtual environment using Pip or easy_install. If you want to remove this virtual environment, you can simply run the following:

$ rmvirtualenv test 

Enjoy!

References:

Homebrew: http://mxcl.github.com/homebrew/
pip: http://pip.openplans.org/
Virtualenv: http://virtualenv.openplans.org/
Virtualenvwrapper: http://www.doughellmann.com/projects/virtualenvwrapper/
and just in case... Python: http://www.python.org/