Upscaling ExaHyPE – on each and every core

Li, Baojiu; Schulz, Holger; Tuft, Adam; Weinzierl, Tobias; Zhang, Han

doi:10.5281/zenodo.7888492

Upscaling ExaHyPE – on each and every core

Li, Baojiu; Schulz, Holger; Tuft, Adam; Weinzierl, Tobias; Zhang, Han

Authors

Professor Baojiu Li baojiu.li@durham.ac.uk
Professor

Holger Schulz

Adam Tuft adam.s.tuft@durham.ac.uk
PGR Student Doctor of Philosophy

Professor Tobias Weinzierl tobias.weinzierl@durham.ac.uk
Professor

Han Zhang han.zhang3@durham.ac.uk
Post Doctoral Research Associate

Abstract

We study a MPI+multithreaded PDE solver for hyperbolic partial differential equations. Each thread per rank handles a subdomain of the computational domain identified by a segment of a space-filling curve. The threads spawn additional tasks which should be used to compensate for ill-balancing between the threads running in fork-join mode. Our studies show that this tasks-over-BSP paradigm is not properly supported in some OpenMP runtimes, leads to NUMA pollution and is vulnerable to tiny tasks. It also suffers from many memory movements. Once we replace user data with smart pointers and hence avoid unnecessary copying, we propose to add a NUMA-aware queuing system on top of OpenMP, to batch multiple tasks into meta tasks which can spread out over idle cores. Many of these techniques are fixes to current OpenMP runtime implementations and we expect them to become unnecessary as the OpenMP runtimes evolve. The insights thus have pathfinding character.

Citation

Li, B., Schulz, H., Tuft, A., Weinzierl, T., & Zhang, H. (2023). Upscaling ExaHyPE – on each and every core. ARCHER2

Report Type	Technical Report
Online Publication Date	May 2, 2023
Publication Date	2023
Deposit Date	May 5, 2023
Publicly Available Date	May 5, 2023
DOI	https://doi.org/10.5281/zenodo.7888492
Additional Information	Publisher: ARCHER2 Type: monograph Subtype: technical_report