You are here: Home Resources Computers Historical Computers at PDC Hebb Octave on Blue Gene MPITB and Octave on Blue Gene/L

MPITB and Octave on Blue Gene/L

This is how we installed MPITB and Octave on BlueGene/L

MPITB and Octave on Blue Gene/L

Nils Smeds, IBM Sweden. <nils.smeds> at <>
Mattias Claesson, PDC, KTH. <slas> at <>

The information in this document is provided as is. No warranties or support is given for the procedure described here. You are free to use this information and the patches here with no obligations. The GNU Octave and MPITB packages have their own licensing agreement. This document does not change or modify your rights and obligations in respect to those packages. To summarize, this has worked for us (to the extent that we have tested it). It may also work for you. If it does we would be happy to hear about it. If it does not we are interested in hearing that too.

GNU Octave is a tool for rapid development of mathematical methods and algorithms that is widely used in a number of disciplines.

MPITB is an add-on to GNU Octave that enables the Matlab user to access MPI library routines from Octave. It also provides a lot of interesting glue code to allow Matlab data structures (such as matrices) to be used directly in MPI calls.

All in all this means that these two code sets allow the user to write parallel Octave code using a well known message passing paradigm.

This document describes one way to compile octave on the IBM Blue Gene/L supercomputer and how to modify the MPITB sources to work with the IBM Blue Gene/L MPI implementation.

1. Getting the source code

GNU Octave source code is needed and can be found at We have tried versions 2.9.12, 3.0.0 and (beta release) 3.0.1b.

Download the Octave source code to a suitable directory where you can build the package. Unpack it and you will get a source tree <SRC_ROOT>/octave-x.y.z

MPITB can be downloaded from There is no pure source code download so you will need to download a pre-compiled package. The source code is included with these packages. We used the MPITB beta for FC6 Octave 2.9.12 with LAM 7.1.3 or OpenMPI 1.2.3.

Download this package to the same directory where you put the Octave tar file. Unpack it and you will get a second tree <SRC_ROOT>/mpitb

Compiling Octave

We recommend that you first compile Octave without MPI support. This will make it easier to find out where things go wrong (if they do at all).

Octave it built using a standard "configure and make" process. However, the cross-compilation environment on BlueGene causes the Octave configure to fail. Also you need to do some minor touches to the configure set-up as explained below.

  1. Enter the octave source tree: cd <SRC_ROOT>/octave-x.y.z
  2. Download the enclosed bash script to the base of the source tree.
  3. Edit the bash script to your environment. The following lines would typically be changed:
    • installdir= path where you want the final package installed
    • LIBS= How to link with your ESSL library and OpenMP stubs (needed by ESSL).
    • LIBS= Where to find your xlf libraries
    • LIBS= How to link against your MASS library
  4. Run the script. This takes a while since this will go through the configure process. We recommend piping the output to a file for later reference:
    ./ 2>&1 | tee PDC-config.log
  5. Edit the Makeconf file:
    • At two places you will find -lnss_dns in the definition of the LIBS variable. Make sure that there is a -lc before the nss_dns entry. If not, add it manually.
  6. Build the application. We recommend piping the output to a file for later reference:
    make 2>&1 | tee MAKE.log
  7. The build process will eventually fail. If it fails in the DOCSTRINGS section you are in luck. You have successfully built Octave. If you have not gotten a file src/octave somethiing went wrong and you need to look in the log files above to try to find out what went wrong.
  8. You are now in a position to test the serial Octave.
    •  make install
      The install will copy files to the installation directory. It will again fail when it tries to generate doc-strings.
    • Create a batch script that does the following:
    • Executes in <SRC_ROOT>/octave-x.y.z/test
    • Sets environment variable TMPDIR to a directory writeable by the program
    • <$installdir>/bin/octave --norc --silent --no-history ./fntests.m .
      Note the final period on the command line above.
    • For obvious reasons you should run it using only one BG/L processor. An example of such a batch file is available.
    The fntest.log file together with the output generated by the test program will serve as a guide to you if the build is acceptible. Some failures are expected. E.g. BG/L does not support process spawning, I/O Pipes or the system() call. Also some tests fail as the accuracy tests are too restrictive. The tests assume that the operations can be done down to machine precision (eps). In reality one most allow for some fluctuation in the last few bits.

Adding MPI to the mixture

You are now ready for adding the MPI layer to the source code
  1. Make certain that you are standing in the base of the octave build tree
  2. Copy the relevant source code files from the mpitb distribution.
    cp ../mpitb/src/{*.h,MPI_*} src/DLD-FUNCTIONS
  3. Apply the MPICH porting patch to the MPITB files.
    • Test the patching first:
      patch -p0 --dry-run < mpitb-2.9.12-update00.patch
      You should see a number of messages that tells you that a patch to a file is applied successfully.
    • Then you can apply it for real:
      patch -p0 < mpitb-2.9.12-update00.patch
  4. Now add the new modules to the octave build process. In the Makefile in the src directory we need to expand the variable DLD_XSRC variable. Execute the following command:
    (cd src/DLD-FUNCTIONS/; /bin/ls -w 100 -C MPI_*) | awk 'BEGIN {printf("DLD_XSRC +=") } NF { printf(" \\\n\t")}; { printf"%s",$0}; END {print "\n"} '
  5. The output of the above command shall now be added into the file src/Makefile. Locate the line where DLD_XSRC is defined. Below this paragraph add the output above. The result will be something like:
    DLD_XSRC := \
            __gnuplot_raw__.l \
    DLD_XSRC += \
    E.g. you can do this by running the command in the previous item and pipe its output into a file. Then edit src/Makefile using your favourite editor and insert this file at the right location.
  6. If you want you can make ther modifications to the Octave source code as needed. E.g. we use the following patch to get the warning label id printed with Octave warnings.
    patch -p0 < octave-warning.patch
  7. Build the application. We recommend piping the output to a file for later reference:
    make 2>&1 | tee MAKE.log

Testing it

We have below a small example on how to use MPITB with Octave for sending matrices between processes. This is derived from the MPITB tests, but modified to not use the spawning facility of MPI which is not present on Blue Gene/L. The printed output is not structured, instead you will find that processes write output and it get garbled beacuse they are writing simultaneously. The important output is occurences of the string "OK" and "FAIL". Look for those in your output file using, for example, a tool such as grep.


This document has been spin-off of the joint research program between the Karolinska Institute, Royal Institute of Technology and IBM. This research involves the advancement of computational bioscience using highly parallel methods. The research groups involved are part of a consortium that owns a Blue Gene/L supercomputer hebb

This work could not have been completed without the help of a number of people. In particular we would like to thank the following people:

  • Dr Rahul Garg of IBM TJ Watson laboratories for guiding us through the process
  • Drs Bob Walkup, Gheorghe Almasi and Jim Sexton of IBM TJ Watson have also been instrumental in this work.
  • Dr Javier Fernández Baldomero of University of Granada - author of MPITB

Contacting us

Please don't hesitate to contact us at the addresses stated at the top of this page.