General instructions

Where to run

Most Summer School exercises will be run on PDC's CRAY XE6 system Lindgren. A number of nodes from Lindgren have been set aside for Summer School usage. A login node has been setup during the Summer School that you should use:

summer-cray.pdc.kth.se

The only exception to this rule is that GPU computer exercises will be run on:

zorn.pdc.kth.se

How to login

To access PDC's cluster you will be using the Ubuntu computers in the computer rooms Orange and Gul.

• Login on any of the computers in Orange or Gul using your PDC or CSC username and password.
• Make sure that you have received Kerberos tickets and AFS tokens on your local computer by typing:
  klist -Tf.
• Log in on Lindgren's Summer School login node:
  ssh -Y summer-cray.pdc.kth.se
• Wait for the prompt to change which signals that you have a window on Lindgren. Check that your login was successful by typing (in that window):
  klist -Tf

More about the environment on Lindgren

The Cray automatically loads several modules at login.

• heimdal - Kerberos commands
• openafs - AFS commands
• torque -  queuing system commands
• PrgEnv-pgi - PGI compiler scripts cc for C code, CC for C++, ftn for Fortran. All the MPI (cray mpich2) are automatically linked. Also loads and links with some math libraries like Scalapack, lapack and blas.

Running OpenMP programs on Lindgren

To start an OpenMP program on the Lindgren compute nodes you should first set the OMP_NUM_THREADS environment variable using the command.

export OMP_NUM_THREADS=6

You then need to use the aprun command to launch the executable (if you do not use aprun then the command will be started on the login node, which could crash the login node if too many people do it).

To launch the program on the compute nodes use

aprun -n 1 -d 6 ./example.x

The -d option is important as without it ALL the OpenMP threads will run on a single core, instead of across multiple cores, and you will not see any parallel speedup. The number after the -d should be the number of threads you want to use, and agree with the OMP_NUM_THREADS environment variable set above.

If all the compute nodes are busy then your job will be held in a simple queue until one is free.

Running MPI programs on Lindgren

MPI programs are started in a similar way to OpenMP programs, again you must use the aprun command to launch the job on the compute nodes.

aprun -n 16 ./example.x

In this example we will start 16 MPI tasks (there are 16 cores per node on the summer-cray).

If you do not use aprun and try to start your program on the login node then you will get an error similar to

Fatal error in MPI_Init: Other MPI error, error stack:
MPIR_Init_thread(408): Initialization failed
MPID_Init(123).......: channel initialization failed
MPID_Init(461).......:  PMI2 init failed: 1