MPI for Python: Performance improvements and MPI-2 extensions

Lisandro Dalcín; Rodrigo Paz; Mario Storti; Jorge D'Elía

doi:10.1016/j.jpdc.2007.09.005

MPI for Python: Performance improvements and MPI-2 extensions

Lisandro Dalcín^*, Rodrigo Paz, Mario Storti, Jorge D'Elía

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

222 Scopus citations

Abstract

MPI for Python provides bindings of the message passing interface (MPI) standard for the Python programming language and allows any Python program to exploit multiple processors. In its first release, MPI for Python was constructed on top of the MPI-1 specification defining an object-oriented interface that closely followed the MPI-2 C ++ bindings, and provided support for communications of general Python objects. In the latest release, this package is improved to enable direct blocking/non-blocking communication of numeric arrays, and to support almost all MPI-2 features. Improvements in communication performance have been tested in a Beowulf class cluster. Results showed a negligible overhead in comparison to compiled C code. MPI for Python is open source and available for download on the web (http://mpi4py.scipy.org/).

Original language	English (US)
Pages (from-to)	655-662
Number of pages	8
Journal	Journal of Parallel and Distributed Computing
Volume	68
Issue number	5
DOIs	https://doi.org/10.1016/j.jpdc.2007.09.005
State	Published - May 2008
Externally published	Yes

Keywords

High-level languages
MPI
Message passing
Parallel Python

ASJC Scopus subject areas

Software
Theoretical Computer Science
Hardware and Architecture
Computer Networks and Communications
Artificial Intelligence

Access to Document

10.1016/j.jpdc.2007.09.005

Cite this

@article{c4412553e22947d39c9b3f83dcbfabd2,

title = "MPI for Python: Performance improvements and MPI-2 extensions",

abstract = "MPI for Python provides bindings of the message passing interface (MPI) standard for the Python programming language and allows any Python program to exploit multiple processors. In its first release, MPI for Python was constructed on top of the MPI-1 specification defining an object-oriented interface that closely followed the MPI-2 C ++ bindings, and provided support for communications of general Python objects. In the latest release, this package is improved to enable direct blocking/non-blocking communication of numeric arrays, and to support almost all MPI-2 features. Improvements in communication performance have been tested in a Beowulf class cluster. Results showed a negligible overhead in comparison to compiled C code. MPI for Python is open source and available for download on the web (http://mpi4py.scipy.org/).",

keywords = "High-level languages, MPI, Message passing, Parallel Python",

author = "Lisandro Dalc{\'i}n and Rodrigo Paz and Mario Storti and Jorge D'El{\'i}a",

note = "Funding Information: The core of MPI for Python is implemented as an extension module written in C in order to access and take advantage of any other MPI implementation providing a C interface. The exposed Python interface is implemented with Python code defining the relevant class hierarchies, functions, and constants. This higher-level code is supported by the C extension module.",

year = "2008",

month = may,

doi = "10.1016/j.jpdc.2007.09.005",

language = "English (US)",

volume = "68",

pages = "655--662",

journal = "Journal of Parallel and Distributed Computing",

issn = "0743-7315",

publisher = "Academic Press Inc.",

number = "5",

}

TY - JOUR

T1 - MPI for Python

T2 - Performance improvements and MPI-2 extensions

AU - Dalcín, Lisandro

AU - Paz, Rodrigo

AU - Storti, Mario

AU - D'Elía, Jorge

N1 - Funding Information: The core of MPI for Python is implemented as an extension module written in C in order to access and take advantage of any other MPI implementation providing a C interface. The exposed Python interface is implemented with Python code defining the relevant class hierarchies, functions, and constants. This higher-level code is supported by the C extension module.

PY - 2008/5

Y1 - 2008/5

N2 - MPI for Python provides bindings of the message passing interface (MPI) standard for the Python programming language and allows any Python program to exploit multiple processors. In its first release, MPI for Python was constructed on top of the MPI-1 specification defining an object-oriented interface that closely followed the MPI-2 C ++ bindings, and provided support for communications of general Python objects. In the latest release, this package is improved to enable direct blocking/non-blocking communication of numeric arrays, and to support almost all MPI-2 features. Improvements in communication performance have been tested in a Beowulf class cluster. Results showed a negligible overhead in comparison to compiled C code. MPI for Python is open source and available for download on the web (http://mpi4py.scipy.org/).

AB - MPI for Python provides bindings of the message passing interface (MPI) standard for the Python programming language and allows any Python program to exploit multiple processors. In its first release, MPI for Python was constructed on top of the MPI-1 specification defining an object-oriented interface that closely followed the MPI-2 C ++ bindings, and provided support for communications of general Python objects. In the latest release, this package is improved to enable direct blocking/non-blocking communication of numeric arrays, and to support almost all MPI-2 features. Improvements in communication performance have been tested in a Beowulf class cluster. Results showed a negligible overhead in comparison to compiled C code. MPI for Python is open source and available for download on the web (http://mpi4py.scipy.org/).

KW - High-level languages

KW - MPI

KW - Message passing

KW - Parallel Python

UR - http://www.scopus.com/inward/record.url?scp=41449101853&partnerID=8YFLogxK

U2 - 10.1016/j.jpdc.2007.09.005

DO - 10.1016/j.jpdc.2007.09.005

M3 - Article

AN - SCOPUS:41449101853

SN - 0743-7315

VL - 68

SP - 655

EP - 662

JO - Journal of Parallel and Distributed Computing

JF - Journal of Parallel and Distributed Computing

IS - 5

ER -

MPI for Python: Performance improvements and MPI-2 extensions

Abstract

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this