DMCA
Avoiding the disk bottleneck in the data domain deduplication file system (2008)
Cached
Download Links
- [www.usenix.org]
- [www.usenix.org]
- [www.usenix.org]
- DBLP
Other Repositories/Bibliography
| Venue: | In Proceedings of the 6th USENIX Conference on File And Storage Technologies |
| Citations: | 161 - 1 self |
BibTeX
@INPROCEEDINGS{Zhu08avoidingthe,
author = {Benjamin Zhu and Kai Li and Hugo Patterson},
title = {Avoiding the disk bottleneck in the data domain deduplication file system},
booktitle = {In Proceedings of the 6th USENIX Conference on File And Storage Technologies},
year = {2008}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Disk-based deduplication storage has emerged as the new-generation storage system for enterprise data protection to replace tape libraries. Deduplication removes redundant data segments to compress data into a highly compact form and makes it economical to store backups on disk instead of tape. A crucial requirement for enterprise data protection is high throughput, typically over 100 MB/sec, which enables backups to complete quickly. A significant challenge is to identify and eliminate duplicate data segments at this rate on a low-cost system that cannot afford enough RAM to store an index of the stored segments and may be forced to access an on-disk index for every input segment. This paper describes three techniques employed in the production Data Domain deduplication file system to relieve the disk bottleneck. These techniques include: (1) the Summary Vector, a compact in-memory data structure for identifying new segments; (2) Stream-Informed Segment Layout, a data layout method to improve on-disk locality for sequentially accessed segments; and (3) Locality Preserved Caching, which maintains the locality of the fingerprints of duplicate segments to achieve high cache hit ratios. Together, they can remove 99 % of the disk accesses for deduplication of real world workloads. These techniques enable a modern two-socket dual-core system to run at 90 % CPU utilization with only one shelf of 15 disks and achieve 100 MB/sec for single-stream throughput and 210 MB/sec for multi-stream throughput. 1
Keyphrases
disk bottleneck data domain deduplication file system mb sec enterprise data protection high cache new segment redundant data segment accessed segment multi-stream throughput stored segment cpu utilization on-disk index data layout method new-generation storage system duplicate segment locality preserved caching modern two-socket dual-core system summary vector disk-based deduplication storage duplicate data segment significant challenge enough ram tape library low-cost system real world workload input segment crucial requirement compact in-memory data structure stream-informed segment layout high throughput on-disk locality disk access single-stream throughput compact form
