TY - GEN
T1 - Design and Evaluation of Multi-Threaded Optimizations for Individual MPI I/O Operations
AU - Feki, Raafat
AU - Gabriel, Edgar
N1 - KAUST Repository Item: Exported on 2022-06-21
Acknowledgements: Partial support for this work was provided by the National Science Foundation under Award No. SI2-SSI 1663887. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. The authors would also like to thank the Supercomputing Laboratory at King Abdullah University of Science & Technology (KAUST) for providing compute time on the Shaheen II supercomputer for this project.
This publication acknowledges KAUST support, but has no KAUST affiliated authors.
PY - 2022/4/18
Y1 - 2022/4/18
N2 - Todays high-end parallel clusters are architecturally very complex. Most large scale applications nowadays are utilizing multiple parallel programming paradigms to achieve the required scalability, with MPI+threads being the most common approach. Yet, as of today, there is no parallel I/O library that matches this hybrid programming model. File I/O operations are typically executed by a single thread for each process. This paper explores multi-Threaded optimizations for individual MPI I/O operations, an important step towards matching the execution model of modern parallel applications. We describe the changes necessary to the internal processing in the MPI I/O library as well as to the file access phase. We demonstrate the performance improvement of the redesigned functions using multiple benchmarks and on multiple platforms for many scenarios over the original, single-Threaded version.
AB - Todays high-end parallel clusters are architecturally very complex. Most large scale applications nowadays are utilizing multiple parallel programming paradigms to achieve the required scalability, with MPI+threads being the most common approach. Yet, as of today, there is no parallel I/O library that matches this hybrid programming model. File I/O operations are typically executed by a single thread for each process. This paper explores multi-Threaded optimizations for individual MPI I/O operations, an important step towards matching the execution model of modern parallel applications. We describe the changes necessary to the internal processing in the MPI I/O library as well as to the file access phase. We demonstrate the performance improvement of the redesigned functions using multiple benchmarks and on multiple platforms for many scenarios over the original, single-Threaded version.
UR - http://hdl.handle.net/10754/679215
UR - https://ieeexplore.ieee.org/document/9756726/
UR - http://www.scopus.com/inward/record.url?scp=85129659359&partnerID=8YFLogxK
U2 - 10.1109/PDP55904.2022.00027
DO - 10.1109/PDP55904.2022.00027
M3 - Conference contribution
SN - 9781665469586
SP - 122
EP - 126
BT - 2022 30th Euromicro International Conference on Parallel, Distributed and Network-based Processing (PDP)
PB - IEEE
ER -