Document Type

Article

Publication Date

2024

DOI

10.3389/fhpcp.2024.1417040

Publication Title

Frontiers in High Performance Computing

Volume

2

Pages

1417040 (1-21)

Abstract

Hardware heterogeneity is here to stay for high-performance computing. Large-scale systems are currently equipped with multiple GPU accelerators per compute node and are expected to incorporate more specialized hardware. This shift in the computing ecosystem offers many opportunities for performance improvement; however, it also increases the complexity of programming for such architectures. This work introduces a runtime framework that enables effortless programming for heterogeneous systems while efficiently utilizing hardware resources. The framework is integrated within a distributed and scalable runtime system to facilitate performance portability across heterogeneous nodes. Along with the design, this paper describes the implementation and optimizations performed, achieving up to 300% improvement on a single device and linear scalability on a node equipped with four GPUs. The framework in a distributed memory environment offers portable abstractions that enable efficient inter-node communication among devices with varying capabilities. It delivers superior performance compared to MPI+CUDA by up to 20% for large messages while keeping the overheads for small messages within 10%. Furthermore, the results of our performance evaluation in a distributed Jacobi proxy application demonstrate that our software imposes minimal overhead and achieves a performance improvement of up to 40%. This is accomplished by the optimizations at the library level and by creating opportunities to leverage application-specific optimizations like over-decomposition.

Rights

© 2024 Thomadakis and Chrisochoides.

This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International (CC BY 4.0) License. The use, distribution or reproduction in other forums is permitted, provided the original authors and the copyright owners are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

Data Availability

Article states: "The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation."

Original Publication Citation

Thomadakis, P., & Chrisochoides, N. (2024). Runtime support for CPU-GPU high-performance computing on distributed memory platforms. Frontiers in High Performance Computing, 2, 1-21, Article 1417040. https://doi.org/10.3389/fhpcp.2024.1417040

ORCID

0000-0002-4299-570X (Thomadakis), 0000-0003-3088-0187 (Chrisochoides)

Download

Share

COinS