Hands-on Exercise¶
Kerberos¶
Please follow this page for necessary configuration steps for Kerberos.
To create a forwardable Kerberos ticket, type
$ kinit -f <username>@NADA.KTH.SE
To list your Kerberos tickets, type
$ klist -f
The output will look similar to
Credentials cache: FILE:/tmp/krb5cc_1001
Principal: lxin@NADA.KTH.SE
Issued Expires Flags Principal
Aug 6 11:47:33 2018 Aug 6 21:47:29 2018 FIA krbtgt/NADA.KTH.SE@NADA.KTH.SE
Login¶
Please follow this page for necessary configuration steps for login.
On Linux/MacOS, type one of these (use PuTTY on Windows)
$ ssh <username>@beskow.pdc.kth.se
$ ssh -o GSSAPIAuthentication=yes <username>@beskow.pdc.kth.se
After logging in, pwd shows your current directory
$ pwd
/afs/pdc.kth.se/home/u/username
The hostname command shows the hostname of the login node
$ hostname
beskow-login2.pdc.kth.se
The top command shows a snapshot of currently running processes (press q to quit):
$ top
Storage quota¶
Please visit this page to learn the AFS (/afs/pdc.kth.se) and Lustre (/cfs/klemming) file systems.
Check your AFS home usage
$ fs lq ~
Check your /cfs/klemming usage
$ lfs quota -u $USER /cfs/klemming
Project¶
The projinfo command accesses a database that contains logs of all allocations
$ projinfo
Modules¶
List all available modules
$ module avail
List all loaded modules
$ module list
List all modules matching a pattern (e.g. PrgEnv)
$ module avail PrgEnv
Swap programming environments (i.e. compiler environments)
$ module swap PrgEnv-cray PrgEnv-gnu
After swapping these modules, the compiler wrappers cc, CC and ftn point to the GNU compilers (instead of the Cray compilers).
Compiling code¶
Clone the summer-school branch of the introduction-to-pdc repository to your klemming directory
$ cd /cfs/klemming/nobackup/${USER:0:1}/$USER
$ module load git
$ git clone -b summer-school \
https://github.com/PDC-support/introduction-to-pdc.git
$ cd introduction-to-pdc/example/
Check compiler version
$ cc --version
gcc (GCC) 4.9.1 20140716 (Cray Inc.)
When using the compiler wrappers cc, CC and ftn, no MPI libraries need to be linked to explicitly
$ cc -o hello_world_mpi hello_world_mpi.c
Compile with OpenMP support
$ cc -fopenmp -o hello_world_openmp hello_world_openmp.c
Submitting jobs¶
Open sbatch_beskow.sh in your favorite editor (nano, emacs or vim, for an introduction to editors see Editing your files) and update SLURM settings
#SBATCH -A edu18.summer
#SBATCH -t 00:05:00
#SBATCH --nodes=2
#SBATCH --reservation=summer-2018-08-14
Here, -A sets the allocation/project ID, -t sets the requested wall time of the job, and --nodes sets the number of nodes that the job will run on. Note that both the -A and --reservation options should be given (otherwise the job might be very slow to start).
The srun command in sbatch_beskow.sh requests that the job will run 64 MPI processes in total (via -n option)
srun -n 64 ./hello_world_mpi
Submit the job to the SLURM queue
$ sbatch sbatch_beskow.sh
Monitor your jobs in the SLURM queue
$ squeue -u <username>
The output is written to a default filename
$ cat slurm-2559495.out
Exercise: repeat the above steps for the file sbatch_tegner.sh, and submit it on Tegner. Note that on Tegner one uses the mpirun command instead of srun.
Running interactively¶
Log back in to Beskow, and request an interactive node for 5 minutes
$ salloc -A edu18.summer --reservation=summer-2018-08-14 -t 00:05:00 -N 1
Compile MPI and OpenMP code
$ cc -o hello_world_mpi hello_world_mpi.c
$ cc -fopenmp -o hello_world_openmp hello_world_openmp.c
Run MPI code
$ srun -n 32 ./hello_world_mpi
Run OpenMP code (the environment variable OMP_NUM_THREADS needs to be set as 32)
$ export OMP_NUM_THREADS=32
$ srun -n 1 ./hello_world_openmp
Note that the srun command sends the job to the compute node. Omitting srun means the calculation will run on the login node. Check this with the following command
$ hostname
$ srun -n 1 hostname
Basic Linux commands¶
Please refer to Basic Linux for new HPC users and Editing your files.