Reclaiming Space from Duplicate Files in a Serverless Distributed File System (2002)
Cached
Download Links
| Venue: | In Proceedings of 22nd International Conference on Distributed Computing Systems (ICDCS |
| Citations: | 61 - 2 self |
BibTeX
@INPROCEEDINGS{Douceur02reclaimingspace,
author = {John R. Douceur and John R. Douceur and Atul Adya and Atul Adya and William J. Bolosky and William J. Bolosky and Dan Simon and Dan Simon and Marvin Theimer and Marvin Theimer},
title = {Reclaiming Space from Duplicate Files in a Serverless Distributed File System},
booktitle = {In Proceedings of 22nd International Conference on Distributed Computing Systems (ICDCS},
year = {2002}
}
Years of Citing Articles
Bookmark
 |
 |
 |
 |
 |
 |
OpenURL
Abstract
The Farsite distributed file system provides availability by replicating each file onto multiple desktop computers. Since this replication consumes significant storage space, it is important to reclaim used space where possible. Measurement of over 500 desktop file systems shows that nearly half of all consumed space is occupied by duplicate files. We present a mechanism to reclaim space from this incidental duplication to make it available for controlled file replication. Our mechanism includes 1) convergent encryption, which enables duplicate files to coalesced into the space of a single file, even if the files are encrypted with different users' keys, and 2) SALAD, a SelfArranging, Lossy, Associative Database for aggregating file content and location information in a decentralized, scalable, fault-tolerant manner. Large-scale simulation experiments show that the duplicate-file coalescing system is scalable, highly effective, and fault-tolerant.
