Python – Building on OS X

Since OS X comes with Python, if you need to build a specific version on your Mac the required bits are likely to be there already. Running “make test” might reveal a few unnoticed absences though.

OS X comes with Python installed and ready go to – just type python in the terminal window and you’re away.

It’s Python 2.7 on my OS X 10.7 build, which is fine if that’s the version you need. If you’re a Python 3 developer, or need a specific Python 2 build for a particular project, you’ll either need to find a precompiled package for your Mac or build the version you need by downloading the source bundle from python.org.

Fortunately, since the Mac comes with Python, most of the bits you need to build a specific version will be there already – if you’re not a developer the only bit you might not have yet are the Apple XCode development tools.

Like many open source tools though, Python configures itself to the libraries available on your system. If anything non-critical is missing it’ll just get skipped. So you might find your successful build isn’t as complete as you expected.

Building Python

Once you’ve downloaded the source bundle for the version you need and you’ve extracted somewhere on your Mac, all you need to do is run the usual configure -> make -> make install process. Chances are it’ll run through without complaint.

./configure
make
make install

If you want to keep multiple versions around you might find the –prefix option useful to put your build in a specific versioned sub-folder

./configure --prefix /usr/local/python-3.3.1
make
make install

Once built, ensure the bin directory is on your PATH and you’re good to go.

export PATH=/usr/local/python-3.3.1/bin
python

Maybe…

Testing Python

And, in fairness, you probably are good to go; but even if the build completed without issues you may still find some bits missing, and depending on the project you’re working on they could be bits you need.

Check the tail end of the output from make and you might see messages like this:-

Python build finished, but the necessary bits to build
these modules were not found:

_bsddb             dl                 gdbm
imageop            linuxaudiodev      ossaudiodev
spwd               sunaudiodev        _lzma

To find the necessary bits, look in setup.py in detect_modules()
for the module's name.

Some of these are obviously not going to be an issue – sunaudiodev isn’t going to hurt me on my Mac. But some may be relevant. A good way to check for what’s expected to be there on your platform, but isn’t, is to run a make test before your make install.

Depending on your hardware and your Python version the test pack can easily take anything from 5 to 15 minutes to run – a nice opportunity to grab a coffee or head over to xvideos.com for a little light relief.

When it’s finished you’ll have a report like the following from my Python 2.7.4 build):-

357 tests OK.
1 test failed:
    test_urllib2_localnet
34 tests skipped:
    test_al test_bsddb test_bsddb3 test_cd test_cl test_codecmaps_cn
    test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr
    test_codecmaps_tw test_curses test_dl test_epoll test_gdb
    test_gdbm test_gl test_imageop test_imgfile test_largefile
    test_linuxaudiodev test_msilib test_ossaudiodev test_smtpnet
    test_socketserver test_startfile test_sunaudiodev test_timeout
    test_tk test_ttk_guionly test_urllib2net test_urllibnet
    test_winreg test_winsound test_zipfile64
4 skips unexpected on darwin:
    test_dl test_readline test_tk test_ttk_guionly

Or this from my Python 3.3.1 build:-

353 tests OK.
2 tests failed:
    test_urllib2_localnet test_urllib2net
1 test altered the execution environment:
    test_warnings
15 tests skipped:
    test_curses test_dbm_gnu test_devpoll test_epoll test_gdb
    test_largefile test_lzma test_msilib test_ossaudiodev
    test_startfile test_tk test_ttk_guionly test_winreg test_winsound
    test_zipfile64
3 skips unexpected on darwin:
    test_lzma test_tk test_ttk_guionly

Note that Python 3 re-runs any failed tests in verbose mode after issuing this report, so if you had test failures you might need to scroll back a bit to see this report.

The most interesting bits of this report are any tests failed and any skips unexpected since these are tests the build expects to work okay on your platform. Depending on whether you need that functionality or not you might be able to ignore them, but they warrant a little investigation to avoid problems later. Here are some suggestions for the ones ones I hit

test_readline

The GNU readline library is used by the Python interpreter to provide command history editing. As such it’s largely non-functional in terms of your Python applications.It doesn’t come with OS X but if you want it, it’s pretty easy to build with the instructions here.

test_dl

The DL library supports direct access to native operations in shared libraries and is deprecated in Python 2.7.x – your time might be better spent changing your code if this would be a problem.

test_tk & test_ttk_guionly

TCL/TK provides GUI features for Python programs so might be important to you. It comes with OS X too, so you might be surprised to see this test unexpectedly skipped.Fortunately it’s almost certainly a false-positive.Scrolling back through the test output you’ll probably see lines like the following:-

test_tk skipped -- cannot run without OS X gui 
test_ttk_guionly skipped -- cannot run without OS X gui process

These are generated by an OS X specific pre-condition check that runs as part of the test. Unfortunately the check doesn’t always work correctly depending on your versions.

Commenting out this precondition check may well remove the unexpected skip. On Python 2.7.x edit Lib/lib-tk/test/runtktests.py or with Python 3.x edit Lib/tkinter/test/support.py. You’ll see an if statement like this:-

 if sys.platform == 'darwin':

Comment out this if…else clause and reduce the indentation on the content of the else clause, then re-run your tests. Hopefully the unexpected skips will disappear.

However, the tests won’t have actually run. They’ve just become “expected skips”.

test_tk skipped -- Use of the 'gui' resource not enabled
test_ttk_guionly skipped -- Use of the 'gui' resource not enabled

This is because GUI tests are disabled by default, since the flickering boxes on the display can be distracting, and because they won’t work if you don’t have access to a graphical console – such as running the build via ssh on a remote server.

If you really, really want to run this test you can set the following environment variable (Python 3.x) to enable GUI tests:-

export EXTRATESTOPTS="-u all,-largefile,-audio"

Or this for Python 2.x:-

export EXTRATESTOPTS="-u gui"

And run make test again.

test_lzma

You may see this from Python 3.3.x which has a new library supporting LZMA compression. Chances are your Mac doesn’t have a system library to match.If you think you’ll need LZMA compression support, or want to make sure it’s there, follow the instructions here.

test_urllib2_localnet & test_urllib2net

These tests exercise URL access (FTP, HTTP, etc.) using some well-known remote addresses. If they’re failing there’s a good chance it could be because your ISP is interceding with a “helpful” error page that the test isn’t expecting to see.If this is the case you’re not going to get these tests to succeed.

f you want a bit more confidence though, follow the stack trace to find the specific test URLs that are causing an issue and stick them in your web-browser – if you see an ISP specific page that’s probably your problem.

And if you want a clean test run, you can comment the offending URL tests out.

If you’re seeing this one with Python 3.3.x

ERROR: test_ftp (test.test_urllib2net.OtherNetworkTests)

Editing Lib/test/test_urllib2net.py and commenting out the second URL on line 112 might clear it for you.

def test_ftp(self):
    urls = [
        'ftp://ftp.kernel.org/pub/linux/kernel/README',
        #'ftp://ftp.kernel.org/pub/linux/kernel/non-existent-file',
        #'ftp://ftp.kernel.org/pub/leenox/kernel/test',
        'ftp://gatekeeper.research.compaq.com/pub/DEC/SRC'
        '/research-reports/00README-Legal-Rules-Regs',
    ]
    self._test_urls(urls, self._extra_handlers())

If you’re seeing the following:-

FAIL: test_bad_address (test.test_urllib2_localnet.TestUrlopen)

Or this with Python 2.x

test test_urllib2_localnet failed

The stack trace will point you at the offending line and associated URL. There’s not much you can do for this one other than comment out the test if you’re really anal about a clean make test run.