TY - GEN
T1 - Continuous Integration for HPC with Github Actions and Tapis
AU - Pachev, Benjamin
AU - Stuart, Georgia
AU - Dawson, Clint
N1 - KAUST Repository Item: Exported on 2022-09-14
Acknowledgements: This work used the Extreme Science and Engineering Discovery Environment (XSEDE) [18], which is supported by National Science Foundation grant number ACI-1548562. The authors would like to gratefully acknowledge the use of XSEDE allocation TGDMS080016N, which provided access to the Jetstream and Stampede2 supercomputers at the Texas Advanced Computing Center and Indiana University. This project was supported by a Competitive Research Grant (CRG) from King Abdullah University of Science and Technology (KAUST) entitled “Detecting, tracking and forecasting for hazardous spills in the Red Sea” In addition, we thank Dr. Christopher Simmons for many conversations on CI/CD in HPC.
This publication acknowledges KAUST support, but has no KAUST affiliated authors.
PY - 2022/7/8
Y1 - 2022/7/8
N2 - Continuous integration and deployment (CICD) are fundamental to modern software development. While many platforms such as GitHub and Atlassian provide cloud solutions for CICD, these solutions don’t fully meet the unique needs of high performance computing (HPC) applications. These needs include, but are not limited to, testing distributed memory and scaling studies, both of which require an HPC environment. We propose a novel framework for running CICD workflows on supercomputing resources. Our framework directly integrates with GitHub Actions and leverages TACC’s Tapis API for communication with HPC resources. The framework is demonstrated for PYthon Ocean PArticle TRAcking (PYOPATRA), an HPC application for Lagrangian particle tracking.
AB - Continuous integration and deployment (CICD) are fundamental to modern software development. While many platforms such as GitHub and Atlassian provide cloud solutions for CICD, these solutions don’t fully meet the unique needs of high performance computing (HPC) applications. These needs include, but are not limited to, testing distributed memory and scaling studies, both of which require an HPC environment. We propose a novel framework for running CICD workflows on supercomputing resources. Our framework directly integrates with GitHub Actions and leverages TACC’s Tapis API for communication with HPC resources. The framework is demonstrated for PYthon Ocean PArticle TRAcking (PYOPATRA), an HPC application for Lagrangian particle tracking.
UR - http://hdl.handle.net/10754/679937
UR - https://dl.acm.org/doi/10.1145/3491418.3535124
U2 - 10.1145/3491418.3535124
DO - 10.1145/3491418.3535124
M3 - Conference contribution
BT - Practice and Experience in Advanced Research Computing
PB - ACM
ER -