DMCA
Design Alternatives for Implementing Fence Synchronization in MPI-2 One-sided Communication for InfiniBand Clusters ∗
| Citations: | 3 - 0 self |
BibTeX
@MISC{Santhanaraman_designalternatives,
author = {G. Santhanaraman and S. Narravula and A. Mamidala},
title = {Design Alternatives for Implementing Fence Synchronization in MPI-2 One-sided Communication for InfiniBand Clusters ∗},
year = {}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Scientific computing has seen an immense growth in recent years. The Message Passing Interface (MPI) has become the de-facto standard for parallel programming model for distributed memory systems. As the system scale increases, application writers often try to increase the overlap of computation and communication. The MPI-2 standard expanded MPI to include one-sided communication semantics that has the potential for overlapping computation with communication. In this model synchronization between processes needs to be done explicitly to ensure completion before using the data. Fence is one of the mechanisms of providing such synchronization in the one-sided model. In this paper, we study a set of different alternatives for designing the fence synchronization mechanisms. We analyze the various trade-offs of these designs on networks like Infini-Band that provide Remote Direct Memory Access (RDMA) capabilities. We propose a novel design for implementing fence synchronization that uses RDMA write with Immediate mechanism (Fence-Imm-RI) provided by InfiniBand networks. We then characterize the performance of different designs with various one-sided communication pattern microbenchmarks for both latency as well as overlap capability. The new Fence-Imm-RI scheme performs the best in scenarios that require low synchronization overhead as well as good overlap capability (close to 90 % overlap for large messages) as opposed to the other designs that can provide either low synchronization overhead or good overlap capability. 1
Keyphrases
infiniband cluster implementing fence synchronization mpi-2 one-sided communication design alternative low synchronization overhead good overlap capability various trade-off recent year infiniband network model synchronization different design system scale increase different alternative novel design scientific computing immense growth new fence-imm-ri scheme fence synchronization de-facto standard various one-sided communication pattern microbenchmarks large message distributed memory system rdma write one-sided model mpi-2 standard overlap capability application writer remote direct memory access fence synchronization mechanism one-sided communication semantics process need immediate mechanism parallel programming model
