Skip to main content

About the Dardel HPC system

Dardel is an HPE Cray EX supercomputer which features a CPU partition (suitable for a broad range of computational applications) and a GPU partition (intended for the most computationally demanding applications). There is also an associated storage system. This NAISS flagship system was installed at PDC between 2021 and 2023.

In 2019, the Swedish National Infrastructure for Computing (SNIC) decided to invest in a new high-performance computing (HPC) system for Swedish researchers, which would be hosted at PDC. After the initial decision, SNIC also funded several expansions to the system to better serve Swedish research needs over a longer time frame. The new flagship system is an HPE Cray EX supercomputer, called Dardel, which was installed at PDC between 2021 and 2023. At the start of 2023, the National Academic Infrastructure for Supercomputing in Sweden ( NAISS ) took over from SNIC, and PDC became a key associate of NAISS. NAISS is now responsible for the operation of Dardel.

If you are interested in seeing what is inside the HPE Cray EX supercomputer systems in general, you can explore here .

The Dardel system at PDC is housed in three rows of cabinets:

  • the back row (shown on the left in the image above), which was installed in 2021 and contains most of the CPU partition of the system,
  • the front row (shown on the right in the image above), which was installed in 2023 and contains the graphics processing unit (GPU) partition of the system plus additional CPUs, and
  • an air-cooled row with the parallel (Lustre-based) file storage system, which is located behind the compute cabinets.

Installation of the back row of the computational part of the system, along with the storage system, started in August 2021. The back row of the main system has been available for research since December 2021. The front row, which includes the GPU partition, was installed in late 2022. The GPU partition has been open to researchers since the spring of 2023.

System

CPU partition

Each compute node has two AMD EPYC Zen2 2.25 GHz 64-core processors, which means that each compute node has a total of 128 physical CPU cores. Two virtual hardware threads are enabled on each physical CPU core, which means that each compute node has a total of 256 logical/virtual cores.

There are several memory sizes in the CPU partition.

  • 700 × 256 GB (NAISS thin nodes)
  • 268 × 512 GB (NAISS large nodes)
  • 8 × 1024 GB (NAISS huge nodes)
  • 10 × 2048 GB (NAISS giant nodes)
  • 36 × 256 GB (KTH industry/business research nodes)
  • 248 × 512 GB (KTH industry/business research nodes)

This gives a total of 1270 nodes.

GPU partition

The GPU partition comprises 56 GPU nodes, each of which is configured with

  • one AMD EPYC processor with 64 cores (which is a special version of a 64-core AMD CPU that is known as the 7A53 (Trento) and only available from HPE),
  • 512 GB of shared fast HBM2E memory (which is cache-coherent and hence simplifies programming), and
  • four AMD Instinct MI250X GPUs (with an impressive performance of up to 95.7 TFLOPS in double precision when using special matrix operations) that are connected by AMD Infinity Fabric® links as shown in the figure below.
Dardel GPU node architecture

Interconnect

  • HPE Slingshot (ethernet-based) using Dragonfly topology
  • 200 Gb/s since March 2023
High-speed network Slingshot topology as used in the Dardel system

Performance

The LINPACK performance of the CPU partition is 4.08 Petaflops and for the GPU partition, it is 8.26 Petaflops.

Usage

  • available for academic research via NAISS
  • open for business and industrial research projects
  • operation currently guaranteed to be funded till Q3 2026