Compiling Python, NumPy and SciPy with Intel Compilers and Intel MKL

Compute clusters are shared machines so they often contain very outdated versions of software in the name of backward compatibility. While the environment modules project goes a long way of alleviating this problem, I often encounter problems with the Python installation on clusters that I'm working on.

In such a situation, it's often cleanest to just start over with your own mini-installation of Python and all packages that you have control over but that will open a whole new can of snakes1. Since I want to spare you the trial-and-error, I've decided to write things up for you.

Note: This guide assumes that you have a 64-bit system with the Intel compilers installed with binaries icc, icpc, ifort, xiar and xild available in your path and the Intel MKL reachable via a correctly configured $LD_LIBRARY_PATH and $INCLUDE. You will not need root permissions to install the software and all installation will be performed relative to a $PREFIX.

Let's start by setting that prefix variable - a good choice is $HOME/.local:

export PREFIX=~/.local  

You also want to make sure that the following is sourced when you log in (by placing it in your .bashrc, .zshrc, etc.):

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/.local/lib  
export C_INCLUDE_PATH=$C_INCLUDE_PATH:$HOME/.local/include  
export PATH=$HOME/.local/bin:$PATH

export CC=icc  
export CXX=icc  
export FC=ifort  

Now we're ready to roll! 🤘

Python 3.5.1

Grab a Python tarball from the official webpage, extract it and change into the new directory. Then, run:

./configure --prefix=$PREFIX --with-computed-gotos --without-gcc \
    --with-libm=-limf --with-cxx-main=`which icpc` --with-threads \
    --enable-ipv6 --with-signal-module \
    CC=icc CXX=icpc LD=xild AR=xiar \
    LIBS="-lpthread -limf -lirc" \
    CFLAGS="-O3 -fp-model strict -fp-model source -xHost -ipo -prec-div -prec-sqrt" \
    LDFLAGS="-ipo" \
    CPPFLAGS="" \
    CPP="icc -E"
make  
make install  

This will compile Python2 and install it into the $PREFIX/bin and $PREFIX/lib directories. If you want your custom-compiled version of Python be available under the python and pip, you can create some symlinks:

ln -s $PREFIX/bin/python{3,}  
ln -s $PREFIX/bin/pip{3,}  

We can now check if things are working:

which python  
~/.local/bin/python
python  
Python 3.5.1 (default, Mar 15 2016, 11:12:26)  
[GCC Intel(R) C++ gcc 4.8 mode] on linux
Type "help", "copyright", "credits" or "license" for more information.  
>>>

NumPy 1.10.4

Next up is NumPy! Grab a tarball from PyPi, extract and change into the new directory.

In order to use Intel MKL with NumPy, we'll have to create a site.cfg file from the template at site.cfg.example:

cp site.cfg{.example,}  

Now edit the site.cfg to reflect the location of the MKL on your system in library_dirs and include_dirs:

[mkl]
library_dirs = /opt/intel/compilers_and_libraries_2016/linux/mkl/lib/intel64  
include_dirs = /opt/intel/compilers_and_libraries_2016/linux/mkl/include  
mkl_libs = mkl_rt  
lapack_libs =  

Now we can compile and install:

python setup.py build --compiler=intelem  
python setup.py install  

Again, we check if things are working correctly:

import numpy as np  
print(np.ones((4, 4)) + np.eye(4))  
[[ 2.  1.  1.  1.]
 [ 1.  2.  1.  1.]
 [ 1.  1.  2.  1.]
 [ 1.  1.  1.  2.]]

SciPy 0.17.0

Finally, compiling and installing SciPy is straightforward. Again, grab a tarball from PyPi, extract and change into the new directory. Then run:

python setup.py build --compiler=intelem --fcompiler=intelem  
python setup.py install  

More packages with pip

If you need more packages for your installation, you should be able to install them using the pip command bundled with Python 3 by default. Packages will be placed relative to your PYTHONHOME, or more specifically in $PREFIX/.local/lib/python3.5/site-packages.

For example, pip install six

Sources

  1. Horrible pun intended.

  2. Since you're probably interested in writing numerical code, we're enabling aggressive optimization but tell the compiler to be careful with floating point optimizations that might change the numerics.