How to use FEniCS

Software	Version	Cluster
FEniCS	0.7.3	Dardel

Singularity is available as a module on Dardel and should be loaded in your job file to make Singularity executable available.

Running in Batch mode

Example job script for using dolfinx/0.7.3 on 2 nodes.

#!/bin/bash -l
# The -l above is required to get the full environment with modules

# Set the allocation to be charged for this job
#SBATCH -A <your allocation>

# The name of the script is specified
#SBATCH -J my_job

# Recive email notifications with any state change
#SBATCH --mail-type=ALL

# 10 hour wall-clock time will be given to this job
#SBATCH -t 10:00:00

# Set the partition for your job.
#SBATCH -p <partition>

# Number of nodes
#SBATCH --nodes=2

# Number of MPI processes per node
#SBATCH --ntasks-per-node=128

# Output and error files
#SBATCH -e error_file.e%J
#SBATCH -o output_file.o%J

# Load required modules
module load PDC
module load singularity

export OMP_NUM_THREADS=1
export LD_LIBRARY_PATH=$CRAY_LD_LIBRARY_PATH:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/cray/pe/lib64:/usr/lib64:$LD_LIBRARY_PATH
export SINGULARITYENV_LD_LIBRARY_PATH=$LD_LIBRARY_PATH
echo $SINGULARITYENV_LD_LIBRARY_PATH
export FEniCS_CONTAINER=/pdc/software/resources/sing_hub/dolfinx/0.7.3
echo "Starting at.." ; date

# Run FEniCs
srun --mpi=pmi2 --ntasks=64 singularity -s exec -B /etc/libibverbs.d:ro -B /var/spool:/var/spool -B /usr/lib64:/usr/lib64:ro -B /opt:/opt:ro -B /var/opt:/var/opt:ro $FEniCS_CONTAINER bash -c "python3 <input.py>"

echo "Ending at.." ; date

Disclaimer

PDC takes no responsibility for the correctness of results produced with the binaries. Always evaluate the binaries against known results for the systems and properties you are investigating before using the binaries for production jobs.