R-ADMAD: High Reliability Provision for Large-Scale De-duplication Archival Storage Systems
BibTeX
@MISC{Liu_r-admad:high,
author = {Chuanyi Liu and Yu Gu and Linchun Sun and Bin Yan and Dongsheng Wang},
title = {R-ADMAD: High Reliability Provision for Large-Scale De-duplication Archival Storage Systems},
year = {}
}
Bookmark
 |
 |
 |
 |
 |
 |
OpenURL
Abstract
Data de-duplication has become a commodity component in dataintensive systems and it is required that these systems provide high reliability comparable to others. Unfortunately, by storing duplicate data chunks just once, de-duped system improves storage utilization at cost of error resilience or reliability. In this paper, R-ADMAD, a high reliability provision mechanism is proposed. It packs variablelength data chunks into fixed sized objects, and exploits ECC codes to encode the objects and distributes them among the storage nodes in a redundancy group, which is dynamically generated according to current status and actual failure domains. Upon failures, R-ADMAD proposes a distributed and dynamic recovery process. Experimental results show that R-ADMAD can provide the same storage utilization as RAID-like schemes, but comparable reliability to replication based schemes with much more redundancy. The average recovery time of R-ADMAD based configurations is about 2-6 times less than RAID-like schemes. Moreover, R-ADMAD can provide dynamic load balancing even without the involvement of the overloaded storage nodes.
