Skip to main content
PDC Center for High Performance Computing

About the Dardel HPC system

Dardel is an HPE Cray EX supercomputer which features a CPU partition (suitable for a broad range of computational applications) and a GPU partition (intended for the most computationally demanding applications). There is also an associated storage system. This NAISS flagship system was installed at PDC between 2021 and 2023.

Background to Dardel

In 2019, the Swedish National Infrastructure for Computing (SNIC) decided to invest in a new high-performance computing (HPC) system for Swedish researchers, which would be hosted at PDC. The new system was installed at PDC between 2021 and 2023. It is an HPE Cray EX supercomputer, which is called Dardel. After the initial decision, SNIC also funded several expansions to the Dardel system to better serve Swedish research needs over a longer time frame.

At the start of 2023, the National Academic Infrastructure for Supercomputing in Sweden ( NAISS ) took over from SNIC, and PDC became a key associate of NAISS. NAISS is now responsible for the operation of Dardel.

Dardel installation

The Dardel system consists of a CPU partition, a graphics processing unit (GPU) partition and storage which are housed in three rows of cabinets.

  • The row shown on the left in the image above was installed in 2021 and contains most of the CPU partition of the system plus a liquid-based cooling distribution unit. This row is now the middle row of the whole system.
  • The front row (shown on the right in the image above) was installed in 2023 and contains the graphics processing unit (GPU) partition of the system plus additional CPUs and a liquid-based cooling distribution unit.
  • There are three additional cabinets for storage and management located behind these two rows of compute cabinets. These cabinets are air-cooled and contain the parallel (Lustre-based) file storage system. This rear row was installed along with the back row of the computational part of the system in 2021.

Some photos of the process of installing Dardel can be found here .

Performance of Dardel partitions

Current LINPACK (Rmax) performance of Dardel partitions:

The performance of Dardel far exceeds a typical personal computer (PC) or laptop. The table gives some other comparisons between Dardel and a typical personal computer. (If you are not familiar with terms like nodes and cores, you can read the explanation about the parts of a supercomputer .)

Dardel vs a typical PC
  Dardel (first phase) CPUs PC
Nodes 858 1
Cores 109,824 16
Peak speed 3175 TFLOPS 0.7 TFLOPS
Memory 323,936 GB 16 GB
Energy consumption 630,000 W

100 W

Note: The peak speeds listed in this table are based on node-level high-performance LINPACK benchmark tests, which give a measure of computing power in terms of floating point operations per second (FLOPS).

Dardel system design

 If you are interested in seeing what is inside the HPE Cray EX supercomputer systems in general, you can explore here .

Dardel’s CPU partition

The CPU partition consists of 1278 compute nodes. Each node has two AMD EPYC™ Zen2 2.25 GHz 64-core processors, which means that each compute node has a total of 2 X 64 = 128 physical CPU cores. Two virtual hardware threads are enabled on each physical CPU core, which means that each compute node has a total of 128 X 2 = 256 logical/virtual cores. The nodes in the CPU partition come with a variety of memory sizes as listed below.

  • 700 × 256 GB (NAISS thin nodes)
  • 268 × 512 GB (NAISS large nodes)
  • 8 × 1024 GB (NAISS huge nodes)
  • 18 × 2048 GB (NAISS giant nodes)
  • 36 × 256 GB (KTH industry/business research nodes)
  • 248 × 512 GB (KTH industry/business research nodes)

This gives a total of 700 + 268 + 8 + 18 + 36 + 248 = 1278 nodes. As each node has 128 physical cores, that means the CPU partition overall has 1278 X 128 = 163,584 cores.

Dardel’s GPU partition

The GPU partition comprises 62 GPU nodes, each of which is configured with

  • one AMD EPYC™ processor with 64 cores (which is a special version of a 64-core AMD CPU that is known as the 7A53 (Trento) and only available from HPE),
  • 512 GB of shared fast HBM2E memory (which is cache-coherent and hence simplifies programming), and
  • four AMD Instinct™ MI250X GPUs (with an impressive performance of up to 95.7 TFLOPS in double precision when using special matrix operations) that are connected by AMD Infinity Fabric® links as shown in the figure below.
Dardel GPU node architecture

Dardel’s interconnect

  • HPE Slingshot (ethernet-based) using Dragonfly topology
  • 200 Gb/s since March 2023
High-speed network Slingshot topology as used in the Dardel system

Usage of Dardel