Prefetching and caching are techniques commonly used in I/O systems to reduce latency. Many researchers have advocated the use of caching and prefetching to reduce latency in the Web. We derive several bounds on the performance improvements seen from these techniques, and then use traces of Web proxy activity taken at Digital Equipment Corporation to quantify these bounds. We found that for these traces, local proxy caching could reduce latency by at best 26%, prefetching could reduce latency by at best 57%, and a combined caching and prefetching proxy could provide at best a 60% latency reduction. Furthermore, we found that how far in advance a prefetching algorithm was able to prefetch an object was a significant factor in its ability to reduce latency. We note that the latency reduction from caching is significantly limited by the rapid changes of objects in the Web. We conclude that for the workload studied caching offers moderate assistance in reducing latency. Prefetching can of...

Rights to individual papers remain with the author or the author's employer. Permission is granted for noncommercial reproduction of the work for educational or research purposes. This copyright notice must be included in the reproduced paper. USENIX acknowledges all trademarks herein.

Delta compression, which consists of compactly encoding one le version as the result of changes to another, can improve eciency in the use of network and disk resources. Delta compression techniques are readily available and can result in compression factors of ve to ten on typical data. Managing delta-compressed storage, however, is a dicult task. I will present a system that attempts to isolate the complexity of delta-compressed storage management by separating the task of version labeling from performance issues. I will show how the system integrates delta-compressed transport with delta-compressed storage. Existing tools for managing delta-compressed storage suer from weak le system support. Lack of transaction support is responsible for inecient application behavior. The only atomic operation in the traditional le system forces unnecessary disk activity due to copying costs. I will demonstrate that transaction support can improve application performance and extensibility wit...

A comprehensive versioning file system creates and retains a new file version for every WRITE or other modification request. The resulting history of file modifications provides a detailed view to tools and administrators seeking to investigate a suspect system state. Conventional versioning systems do not efficiently record the many prior versions that result. In particular, the versioned metadata they keep consumes almost as much space as the versioned data. This paper examines two space-efficient metadata structures for versioning file systems and describes their integration into the Comprehensive Versioning File System (CVFS). Journal-based metadata encodes each metadata version into a single journal entry; CVFS uses this structure for inodes and indirect blocks, reducing the associated space requirements by 80%. Multiversion b-trees extend the per-entry key with a timestamp and keep current and historical entries in a single tree; CVFS uses this structure for directories, reducing the associated space requirements by 99%. Experiments with CVFS verify that its current-version performance is similar to that of non-versioning file systems. Although access to historical versions is slower than conventional versioning systems, checkpointing is shown to mitigate this effect.

Delta compression and remote file synchronization techniques are concerned with efficient file transfer over a slow communication link in the case where the receiving party already has a similar file (or files). This problem arises naturally, e.g., when distributing updated versions of software over a network or synchronizing personal files between different accounts and devices. More generally, the problem is becoming increasingly common in many networkbased applications where files and content are widely replicated, frequently modified, and cut and reassembled in different contexts and packagings.

. The Xdelta system implements a technique for archiving and compressing collections of many similar file versions. It stores only the differences between certain versions. I describe and discuss an algorithm for computing file deltas, present measurements, and demonstrate its application to versioned file-archival and efficient file-distribution network protocols. 1 Overview The file delta problem is to compute a small set of instructions for transforming one file into another---one that is expected to be a function of the file's changes, not its content. This technique is well established for versioned file-archival. Though the advantages of using file deltas to transmit changes over a network are clear, specifying and widely deploying such a system efficient enough to justify itself is not as easy as it seems. There are a number of issues to overcome. First, the execution cost of computing and compressing deltas can be prohibitive--a site administrator might rather let everyone...

Duplicate Data Elimination (DDE) is our method for identifying and coalescing identical data blocks in Storage Tank, a SAN file system. On-line file systems pose a unique set of performance and implementation challenges for this feature. Existing techniques, which are used to improve both storage and network utilization, do not satisfy these constraints. Our design employs a combination of content hashing, copy-on-write, and lazy updates to achieve its functional and performance goals. DDE executes primarily as a background process. The design also builds on Storage Tank’s FlashCopy function to ease implementation. 1 We include an analysis of selected real-world data sets that is aimed at demonstrating the space-saving potential of coalescing duplicate data. Our results show that DDE can reduce storage consumption by up to 80 % in some application environments. The analysis explores several additional features, such as the impact of varying file block size and the contribution of whole file duplication to the net savings. 1

This paper is organized into nine sections. We begin by reviewing previous disk activity studies in Section 2. In Section 3, we briefly discuss our data collection and analysis tools, which differ significantly from those used in earlier studies. We describe the different types of computing environments from which we collected data in Section 4. The software written for this paper analyzes the collected data and generates statistics. The simplest analysis mode provides information about daily activity. This is shown in Section 5. Analysis of long-term trends is shown in Section 6. An interesting product from this research is a comparison of the same file system's activity from either the file name view, or from the operating system's underlying numeric index. This comparison is done in Section 7. We summarize our findings in Section 8 and briefly discuss our future research in Section 9.

We present and algorithm for modifying delta compressed files so that the compressed versions may be reconstructed without requiring additional memory or storage space. This allows network clients with limited resources to efficiently update software by downloading delta compressed versions over a network. Delta compression for binary files, compactly encoding a version of data with only the changed bytes from a previous version, may be used to efficiently distribute software over low bandwidth channels, such as the Internet. Traditional methods of rebuilding these delta files require memory or storage space on the target machine for both the old and new version of the file to be reconstructed. With the advent of network computing and Internet set-top boxes, many of these network attached target machines have limited additional scratch space in memory or storage. We provide an algorithm for modifying a delta compressed version file so that it may rebuild the new file version in the spa...

Most applications that access large data objects do so through file systems, but file systems provide an incomplete solution, as they maintain insufficient metadata and do not provide general purpose query engine. Storing large objects in a database addresses these problems, but, for applications that need to update object data, databases are inefficient as they do not provide direct access to data. Additionally, databases often relax the integrity and consistency constraints for large objects, as it the case with objects stored through the Binary Large Object (BLOB) data type. These shortcomings are exacerbated by multiple users or applications that wish to access large objects concurrently. We describe an architecture, based on the Datalink data type, in which large objects in a database are continuously available for read access and can be read and written through a file system interface. Additionally, this system does not relax version management, consistency and recoverability guarantees, as with the BLOB data type. 1.