Installing software using Spack

Spack is a package manager for supercomputers. It can be used for installing scientific software and libraries for your own software development project in an easy way.

Below you can find short information on how to use spack at PDC, but the full documentation for spack is available online at https://spack.readthedocs.io/en/latest/index.html

Please note that while there is a central Spack installation used by PDC staff to install software in /pdc/software/..., you cannot use this yourself to install software as it is write-protected. The Spack philosophy of working is instead that you work with your own Spack installation yourself in your private space. It is possible to link your own installation to the PDC central one and reduce the number of package you need to compile by using the so-called “chaining” functionality, see more information here https://spack.readthedocs.io/en/latest/chain.html.

Installing spack

Installing spack is just a matter of cloning its repository

git clone -c feature.manyFiles=true https://github.com/spack/spack.git

Spack needs to be configured to work properly in the Cray compiler environment. For running Spack on Dardel, we suggest copying the configuration files from our central installation. E.g for using Spack 0.17.0 configured against the Cray programming environment 21.11:

cp -r /pdc/software/21.11/spack/0.17.0/etc/spack/* .

After this you can initialize Spack using the start script. This script will initialize your spack environment and also store the configuration files mentioned above into the spack environment.

. spack/share/spack/setup-env.sh

This will put the spack command in your PATH and also add the Spack-generated module files to your MODULEPATH. You can verify that the basic configuration is working by running spack arch and spack compilers

$ spack arch
cray-sles15-zen2
$ spack compilers
==> Available compilers
-- cce sles15-any -----------------------------------------------
cce@13.0.0

-- gcc sles15-any -----------------------------------------------
gcc@11.2.0  gcc@9.3.0

Finding and listing available software

If you would like to find out what software you can install, use the list command:

$ spack list kokkos
==> 6 packages.
kokkos  kokkos-kernels  kokkos-kernels-legacy  kokkos-legacy  kokkos-nvcc-wrapper  py-pykokkos-base

More details about a specific packages, such as configuration flags and build options can be seen with spack info

$ spack info kokkos
$ spack info kokkos
CMakePackage:   kokkos

Description:
    Kokkos implements a programming model in C++ for writing performance
    portable applications targeting all major HPC platforms.

Homepage: https://github.com/kokkos/kokkos

Maintainers: @DavidPoliakoff @jciesko

Externally Detectable:
    False

Tags:
    e4s

Preferred version:
    3.4.01     https://github.com/kokkos/kokkos/archive/3.4.01.tar.gz
...

Installing software using spack

Installing software using spack in the most basic way is simply:

spack install <software>

If there are other packages or libraries that needs to be installed for this software to work, they are also installed automatically. In general, though, you would like to be more specific, and request a certain version of a software, or maybe a specific compiler to be used. There is a specific syntax for doing that:

Command Option
@ Which version to install/use
% Which compiler to use
^ Add a specific dependency

This is an example of an installation of Amber 18 where it is compiled with GCC version 9.3.0 and a specific ncurses package

spack install amber@18%gcc@9.3.0^ncurses@6.2
...
==> amber: Successfully installed amber-18-js3n7jolbdh7gknbfuiwf5vutoaljckd
Fetch: 1.09s.  Build: 46.92s.  Total: 48.01s.
[+] $HOME/spack/opt/spack/cray-sles15-zen2/gcc-9.3.0/amber-18-js3n7jolbdh7gknbfuiwf5vutoaljckd

It is often useful to do a dry run before installing to see what Spack thinks it needs to install. This can be done with spec command. For example, what will happen when we request LINPACK?

$ spack spec -I hpl
Input spec
--------------------------------
-   hpl

Concretized
--------------------------------
-   hpl@2.3%gcc@11.2.0~openmp arch=cray-sles15-zen2
[+]      ^cray-libsci@21.08.1.2%gcc@11.2.0~mpi~openmp+shared arch=cray-sles15-zen2
[+]      ^mpich@8.1.11%gcc@11.2.0~argobots+fortran+hwloc+hydra+libxml2+pci+romio~slurm~two_level_namespace~verbs+wrapperrpath device=ch4 netmod=ofi pmi=pmi arch=cray-sles15-zen2

Here Spack will compile and install LINPACK using gcc version 11.2.0, MPICH for MPI, and Cray’s LibSci for BLAS and LAPCK. Only the LINPACK package itself will be compiled. Cray Libsci and MPICH are already installed, as indicated by the [+] symbols at the beginning of the lines.

Installing non-downloadable software

In some cases it is not possible to directly download the software you are interested in as the software is restricted by license or other.

In this case you have to download it manually and create folder for it. See instructions at https://spack.readthedocs.io/en/latest/basic_usage.html#non-downloadable-tarballs

Garbage collection

Installing and uninstalling softwares will in the end use up your disk space so it is good practice to do some garbage collection

spack gc

How to execute your software

Software installed by Spack are put in the directory spack/opt/spack/cray-sles15-zen2/<compiler>/<software>. This can be a bit tedious to use every time, so there are two ways to make it more convenient. Spack automatically makes module files for all installed software, which shows up when you do module avail, provided that you can run the Spack initilization script mentioned above. You can then use the module load command in the way as you would do for other software at PDC, but in this case you load a software from your own Spack installation.

$ module avail hpl

--------------------------- /cfs/klemming/scratch/y/ypetla/spack/spack/share/spack/modules/cray-sles15-zen2 ---------------------------
hpl-2.3-gcc-11.2.0-5v2u3sq

Alternatively, you can use Spack’s built-in mechanism for loading software packages, which works similarily to module load:

$ spack load hpl
$ which xhpl
/cfs/klemming/pdc/software/dardel/21.11/spack/spack/opt/spack/cray-sles15-zen2/gcc-11.2.0/hpl-2.3-5v2u3sqa3huu2djhxdtjxbfaoysdwfpy/bin/xhpl

…and the software will be directly available via your command line.