Fixing a broken virtualenv after a python update

So I just updated my Ubuntu box from Ubuntu 12.10 to 13.10 and this left all of my python virtualenv’s unusable. The first indication that something was wrong was that ipython would not work. However digging into the problem all python modules in the virtual env did not work correctly. This guide documents the steps taken to solve the issue. Including re-installing virtualenv and virtualenvwrapper. Installing PIL and its dependent libraries. In the end to get the virtualenvs to work again I had to use the big whacking hammer to remove my old ~/.virtualenvs directory and recreate it in the new environment.

art_donkeykong_jumpman

Fortunately that is exactly what virtualenv is for so the process was fairly painless.

$ ipython
Traceback (most recent call last):
  File "/home/jds/.virtualenvs/dj-rest_framework/bin/ipython", line 5, in 
    from pkg_resources import load_entry_point
  File "/home/jds/.virtualenvs/dj-rest_framework/local/lib/python2.7/site-packages/distribute-0.6.42-py2.7.egg/pkg_resources.py", line 16, in 
    import sys, os, time, re, imp, types, zipfile, zipimport
  File "/home/jds/.virtualenvs/dj-rest_framework/lib/python2.7/re.py", line 105, in 
    import sre_compile
  File "/home/jds/.virtualenvs/dj-rest_framework/lib/python2.7/sre_compile.py", line 14, in 
    import sre_parse
  File "/home/jds/.virtualenvs/dj-rest_framework/lib/python2.7/sre_parse.py", line 17, in 
    from sre_constants import *
  File "/home/jds/.virtualenvs/dj-rest_framework/lib/python2.7/sre_constants.py", line 18, in 
    from _sre import MAXREPEAT
ImportError: cannot import name MAXREPEAT

Well that sucks… how do I fix it… First lets try to reinstall virtualenv and virtualenvwrapper.

$ sudo apt-get remove virtualenvwrapper
$ sudo apt-get remove virtualenv
$ sudo pip install virtualenv
Downloading/unpacking virtualenv
  Downloading virtualenv-1.11.6.tar.gz (1.6Mb): 1.6Mb downloaded
... snip...
$ sudo pip install virtualenvwrapper
Downloading virtualenvwrapper-4.3.tar.gz (92Kb): 92Kb downloaded
... snip...
Downloading virtualenv-clone-0.2.5.tar.gz
... snip...
Downloading stevedore-0.15.tar.gz (348Kb): 348Kb downloaded
... snip...
Successfully installed virtualenvwrapper virtualenv-clone stevedore
... snip...

This did not work.

$ cd ~/.virtualenvs
$ ls -al |grep ^-rwxrwxr-x
# Now remove all of those files
#
$ rm -f pre* post* initialize get_env_details

This did not work either. Time for the big whacking hammer.

$ rm -rf ~/.virtualenvs

Now lets recreate the evn.

$ mkvirtualenv dj15
$ workon dj15
$ pip install ipython

Installing PIL can be challenging at times. It does not have to be if you understand that its basically going to depend on a few extra libs. I feel compelled to mention here that there is an additional package ‘Pillow’ which is intended to make installing PIL easier.

  • TKINTER (Tcl/Tk 8.5 libraries needed)
  • JPEG support
  • ZLIB (PNG/ZIP) support
  • FREETYPE2 support
  • LITTLECMS support

Of these five add-on’s I only care about JPEG, ZLIB, and FREETYPE2. So lets make sure that the supporting libs are in place, and if not install them. Before installing PIL.

$ dpkg --list |grep libjpeg-dev
$ dpkg --list |grep zlib1g-dev
$ dpkg --list |grep libfreetype6-dev

If you don’t have those installed and you want PIL to support them, then you will need to apt-get install them.

$ sudo apt-get install libjpeg-dev zlib1g-dev libfreetype6-dev

Now lets get onto the business of installing PIL.

$ pip install PIL
Could not find any downloads that satisfy the requirement PIL
Some externally hosted files were ignored (use --allow-external PIL to allow).
Cleaning up...
No distributions at all found for PIL
Storing debug log for failure in /home/jds/.pip/pip.log

Oopse could not install PIL… What happened here? After digging into the log and stackoverflow I found JCottons excellent answer to the issue. So do what he says:

$ pip install PIL --allow-external PIL --allow-unverified PIL
... snip ...
    --------------------------------------------------------------------
    PIL 1.1.7 SETUP SUMMARY
    --------------------------------------------------------------------
    version       1.1.7
    platform      linux2 2.7.5+ (default, Sep 19 2013, 13:49:51)
                  [GCC 4.8.1]
    --------------------------------------------------------------------
    *** TKINTER support not available (Tcl/Tk 8.5 libraries needed)
    --- JPEG support available
    --- ZLIB (PNG/ZIP) support available
    --- FREETYPE2 support available
    *** LITTLECMS support not available
    --------------------------------------------------------------------
... snip ...

Alright PIL is installed.

Finally set up a the rest of the env from a requirements file. I have included mine for reference.

Example: ~/workspace/virtualenvs/dj15.txt

django==1.5
# Note: south is included in the django distribution for django version 1.7 and up.
south 
psycopg2
djangorestframework
pygments
git+git://github.com/django-debug-toolbar/django-debug-toolbar.git#egg=django-debug-toolbar
git+git://github.com/playfire/django-debug-toolbar-user-panel.git#egg=django-debug-debug-toolbar-user-panel
git+git://github.com/django-extensions/django-extensions.git#egg=django_extensions
Ipython
python-memcached
PIL
simplejson
poster
requests

Install from that requirements file

$ 
$ pip install -r ~/workspace/virtualenvs/dj15.txt
... snip...
Downloading Django-1.5.tar.gz (8.0MB): 8.0MB downloaded
Downloading psycopg2-2.5.3.tar.gz (690kB): 690kB downloaded
Downloading djangorestframework-2.3.14-py2.py3-none-any.whl (298kB): 298kB downloaded
Downloading Pygments-1.6.tar.gz (1.4MB): 1.4MB downloaded
Downloading/unpacking django-debug-toolbar from git+git:...
Downloading/unpacking django-debug-debug-toolbar-user-panel from git+git:...
Downloading/unpacking django-extensions from git+git...
Downloading South-1.0.tar.gz (97kB): 97kB downloaded
Downloading python-memcached-1.53.tar.gz
Requirement already satisfied (use --upgrade to upgrade): PIL
Downloading simplejson-3.5.3.tar.gz (70kB): 70kB downloaded
Downloading poster-0.8.1.tar.gz
Downloading sqlparse-0.1.11.tar.gz (53kB): 53kB downloaded
Downloading six-1.7.3-py2.py3-none-any.whl
Successfully installed django psycopg2 djangorestframework pygments django-debug-toolbar django-debug-debug-toolbar-user-panel django-extensions south python-memcached simplejson poster requests sqlparse six

All is right with the world again.

A note about South 1.0. South is now rolled into django as of django version 1.7.

Reference: