Efficient memory allocation and data transfer for cluster-based data-intensive applications is a difficult task. Both changes in cluster interconnects and application workloads usually require tuning of the application and network code. We propose separating control and data transfer traffic by accessing data through a DSM-like cluster-wide shared buffer space and only including buffer references in the control messages. Using a generic API for accessing buffers allows for tuning data transfer without changing the application code. A prototype, implemented in the context of a distributed storage system, has been validated with several networking technologies, showing that such a framework can combine performance and flexibility. 1

This paper presents a comprehensive micro-benchmark performance evaluation on using NIC memory in the Mellanox InfiniBand adapter. Three main benefits have been explored, including non-blocking and high performance host/NIC data movement, traffic reduction of the local interconnect, and avoidance of the local interconnect bottleneck. Two case studies have been carried out to show how these benefits can be utilized by applications. In the first case in which the NIC memory is used as intermediate communication buffer for non-contiguous data communication, lower CPU overhead and better latency are attained. In the second case, a common communication building block, communication forwarding chain, has been studied. Our results show that using the NIC memory can achieve a factor of up to 2.2 improvement over the conventional approach. To the best of our knowledge, this is the first such study to demonstrate the benefits of NIC memory in InfiniBand adapter. 1.

Abstract. To leverage high speed interconnects like InfiniBand it is important to minimize the communication overhead. The most interfering overhead is the registration of communication memory. In this paper, we present our analysis of the memory registration process inside the Mellanox InfiniBand driver and possible ways out of this bottleneck. We evaluate and characterize the most time consuming parts in the execution path of the memory registration function using the Read Time Stamp Counter (RDTSC) instruction. We present measurements on AMD Opteron and Intel Xeon systems with different types of Host Channel Adapters for PCI-X and PCI-Express. Finally, we conclude with first results using Linux hugepage support to shorten the time of registering a memory region. 1

Abstract not found