Hands-on Exercise


Please follow this page for necessary configuration steps for Kerberos.

To create a forwardable Kerberos ticket, type

$ kinit -f <username>@NADA.KTH.SE

To list your Kerberos tickets, type

$ klist -f

The output will look similar to

Credentials cache: FILE:/tmp/krb5cc_1001
        Principal: lxin@NADA.KTH.SE

  Issued                Expires             Flags    Principal
Aug  6 11:47:33 2018  Aug  6 21:47:29 2018  FIA    krbtgt/NADA.KTH.SE@NADA.KTH.SE


Please follow this page for necessary configuration steps for login.

On Linux/MacOS, type one of these (use PuTTY on Windows)

$ ssh <username>@beskow.pdc.kth.se
$ ssh -o GSSAPIAuthentication=yes <username>@beskow.pdc.kth.se

After logging in, pwd shows your current directory

$ pwd

The hostname command shows the hostname of the login node

$ hostname

The top command shows a snapshot of currently running processes (press q to quit):

$ top

Storage quota

Please visit this page to learn the AFS (/afs/pdc.kth.se) and Lustre (/cfs/klemming) file systems.

Check your AFS home usage

$ fs lq ~

Check your /cfs/klemming usage

$ lfs quota -u $USER /cfs/klemming


The projinfo command accesses a database that contains logs of all allocations

$ projinfo


List all available modules

$ module avail

List all loaded modules

$ module list

List all modules matching a pattern (e.g. PrgEnv)

$ module avail PrgEnv

Swap programming environments (i.e. compiler environments)

$ module swap PrgEnv-cray PrgEnv-gnu

After swapping these modules, the compiler wrappers cc, CC and ftn point to the GNU compilers (instead of the Cray compilers).

Compiling code

Clone the summer-school branch of the introduction-to-pdc repository to your klemming directory

$ cd /cfs/klemming/nobackup/${USER:0:1}/$USER
$ module load git
$ git clone -b summer-school \
$ cd introduction-to-pdc/example/

Check compiler version

$ cc --version
gcc (GCC) 4.9.1 20140716 (Cray Inc.)

When using the compiler wrappers cc, CC and ftn, no MPI libraries need to be linked to explicitly

$ cc -o hello_world_mpi hello_world_mpi.c

Compile with OpenMP support

$ cc -fopenmp -o hello_world_openmp hello_world_openmp.c

Submitting jobs

Open sbatch_beskow.sh in your favorite editor (nano, emacs or vim, for an introduction to editors see Editing your files) and update SLURM settings

#SBATCH -A edu18.summer
#SBATCH -t 00:05:00
#SBATCH --nodes=2
#SBATCH --reservation=summer-2018-08-14

Here, -A sets the allocation/project ID, -t sets the requested wall time of the job, and --nodes sets the number of nodes that the job will run on. Note that both the -A and --reservation options should be given (otherwise the job might be very slow to start).

The srun command in sbatch_beskow.sh requests that the job will run 64 MPI processes in total (via -n option)

srun -n 64 ./hello_world_mpi

Submit the job to the SLURM queue

$ sbatch sbatch_beskow.sh

Monitor your jobs in the SLURM queue

$ squeue -u <username>

The output is written to a default filename

$ cat slurm-2559495.out

Exercise: repeat the above steps for the file sbatch_tegner.sh, and submit it on Tegner. Note that on Tegner one uses the mpirun command instead of srun.

Running interactively

Log back in to Beskow, and request an interactive node for 5 minutes

$ salloc -A edu18.summer --reservation=summer-2018-08-14 -t 00:05:00 -N 1

Compile MPI and OpenMP code

$ cc -o hello_world_mpi hello_world_mpi.c
$ cc -fopenmp -o hello_world_openmp hello_world_openmp.c

Run MPI code

$ srun -n 32 ./hello_world_mpi

Run OpenMP code (the environment variable OMP_NUM_THREADS needs to be set as 32)

$ export OMP_NUM_THREADS=32
$ srun -n 1 ./hello_world_openmp

Note that the srun command sends the job to the compute node. Omitting srun means the calculation will run on the login node. Check this with the following command

$ hostname
$ srun -n 1 hostname

Basic Linux commands

Please refer to Basic Linux for new HPC users and Editing your files.