Liquid metals and solid performance

Using REANNZ HPC resources, Dr Krista Steenbergen runs powerful computer simulations to study how liquid metals behave at the atomic level.

Research background

At the University of Auckland, Dr Krista Steenbergen uses powerful computer simulations to study how liquid metals behave at the atomic level. These metals play important roles in emerging technologies, from advanced batteries to medical devices.

To understand the metals' structure and electronic behaviour, Krista uses the Vienna Ab Initio Simulation Package (VASP), a widely used software package for atomic scale materials modelling.

Simulation snapshot of a doped liquid metal system with an oxide film and surface regions highlighted.[cite: KG Steenbergen, S Lambie, C Ruffman, N Gaston. "Atomic-Scale Dynamics at the Interface of Doped Liquid Gallium: Contrasting Effects of Gallium Oxide and Vacuum", Small Science 5, 2500153 (2025)] The performance gains achieved through the NeSI consultancy made it feasible to carry out this research at a scale and level of detail that would otherwise have been impractical.

Project challenges

VASP's numerically intensive calculations can leverage the parallel architecture of our national HPC Platform. However, to achieve optimal performance, its settings must match the underlying hardware characteristics. This includes choosing the right mix of MPI processes and OpenMP threads, which can vary depending on the cluster being used.

In 2024, Krista consumed more than 6 million core-hours across the Mahuika and Māui clusters on the national HPC Platform. So, running efficiently is paramount.

What was done

On the Mahuika cluster, each job typically shares resources with other jobs running concurrently. These jobs compete with each other for the resources, which can slow down the job. We observed better performance on Mahuika when requesting exclusive access to nodes.

Additional reasons why sharing a node might cause poor performance of VASP:

VASP solves the Schroedinger equation in reciprocal, Fourier space. The dominant part of the execution time is spent performing Fast Fourier Transforms (FFTs), which convert the wavefunctions between real and momentum space. FFTs involve all-to-all communication and are best parallelized using OpenMP threads.
We found that best performance is achieved by ensuring that the threads reside all within a Core Complex Die (CCD), the fundamental block of the AMD EPYC architecture. There are eight CCDs per socket and two sockets per node. Moreover, if a VASP shares resources with other jobs then the performance will likely be negatively impacted. The performance will vary significantly depending on other running jobs on the node.
Requesting full nodes has the disadvantage that jobs may stay longer in the queue. To avoid that, high and consistent performance can be obtained by binding the processes working on a particular wavefunction to cores belonging to the same NUMA domain (a subset of a node).

The graph plot below demonstrates the benefit of assigning OpenMP threads to the same CCD. The smallest execution time is obtained by using only one CCD. When threads spill over multiple CCDs, the execution time linearly increases and, in addition, becomes highly sensitive to the load on the node, causing up to 5x slowdowns.

Main outcomes

Through investigation and benchmarking, we determined the best VASP performance was obtained when requesting exclusive access to nodes. This also reduced fluctuations in the execution times.
SLURM submission scripts were written to improve the performance by up to five times.

Researcher feedback

The consultancy support was exceptional, and working with John was an absolute pleasure. His deep understanding of both the hardware and the VASP software transformed a computationally prohibitive project into one that was fully achievable. This work enabled large-scale state-of-the-art simulations that led to two high-impact publications and significantly advanced our understanding of doped liquid metal systems.

Krista Steenbergen, University of Auckland

This case study shares some of the technical details and outcomes provided through our Consultancy Service. This service supports projects across a range of domains, with an aim to lift researchers’ productivity, efficiency, and skills in research computing. Get in touch to discuss how our Research Software Engineers and specialist support could help advance your project.