Delta compression, which consists of compactly encoding one le version as the result of changes to another, can improve eciency in the use of network and disk resources. Delta compression techniques are readily available and can result in compression factors of ve to ten on typical data. Managing delta-compressed storage, however, is a dicult task. I will present a system that attempts to isolate the complexity of delta-compressed storage management by separating the task of version labeling from performance issues. I will show how the system integrates delta-compressed transport with delta-compressed storage. Existing tools for managing delta-compressed storage suer from weak le system support. Lack of transaction support is responsible for inecient application behavior. The only atomic operation in the traditional le system forces unnecessary disk activity due to copying costs. I will demonstrate that transaction support can improve application performance and extensibility wit...

Inexpensive storage and more powerful processors have resulted in a proliferation of data that needs to be reliably backed up. Network resource limitations make it increasingly difficult to backup a distributed file system on a nightly or even weekly basis. By using delta compression algorithms, which minimally encode a version of a file using only the bytes that have changed, a backup system can compress the data sent to a server. With the delta backup technique, we can achieve significant savings in network transmission time over previous techniques. Our measurements indicate that file system data may, on average, be compressed to within 10% of its original size with this method and that approximately 45% of all changed files have also been backed up in the previous week. Based on our measurements, we conclude that a small file store on the client that contains copies of previously backed up files can be used to retain versions in order to generate delta files. To reduce the load on ...

Modern software systems for application areas like software engineering, CAD, or office automation are usually highly interactive and deal with rather complex object structures. For the realization of these systems a nonstandard database system is needed which is able to efficiently handle different types of coarse- and fine-grained objects (like documents and paragraphs), hierarchical and non-hierarchical relations between objects (like composition-links and cross-references), and finally attributes of rather different size (like chapter numbers and bitmaps). Furthermore, this database system should support incremental computation of derived data, undo/redo of data modifications, error recovery from system crashes, and version control mechanisms. In this paper, we describe the underlying data model and the functionality of GRAS, a database system which has been designed according to the requirements mentioned above. Furthermore, we motivate our central design decisions concerning its ...

Differential Compression: A Generalized Solution for Binary Files by Randal C. Burns This work presents the development and analysis of a family of algorithms for generating differentially compressed output from binary sources. The algorithms all perform the same fundamental task: given two versions of the same data as input streams, generate and output a compact encoding of one of the input streams by representing it as a set of changes with respect to the other input stream. Differential compression provides a computationally efficient compression technique for applications that generate versioned data and we often expect differencing to produce a significantly more compact file than more traditional compression techniques. The greedy algorithm for file differencing is presented and this algorithm is proven to produce the optimally compressed differential output. However, this algorithm requires execution time quadratic in the size of the input files. We next present an algorithm...

An efficient differencing algorithm can be used to compress version of files for both transmission over low bandwidth channels and compact storage. This can greatly reduce network traffic and execution time for distributed applications which include software distribution, source code control, file system replication, and data backup and restore. An algorithm for such applications needs to be both general and efficient; able to compress binary inputs in linear time. We present such an algorithm for differencing files at the granularity of a byte. The algorithm uses constant memory and handles arbitrarily large input files. While the algorithm makes minor sacrifices in compression to attain linear runtime performance, it outperforms the byte-wise differencing algorithms that we have encountered in the literature on all inputs. I. INTRODUCTION Differencing algorithms compress data by taking advantage of statistical correlations between different versions of the same data sets. Strictly ...

. The Xdelta system implements a technique for archiving and compressing collections of many similar file versions. It stores only the differences between certain versions. I describe and discuss an algorithm for computing file deltas, present measurements, and demonstrate its application to versioned file-archival and efficient file-distribution network protocols. 1 Overview The file delta problem is to compute a small set of instructions for transforming one file into another---one that is expected to be a function of the file's changes, not its content. This technique is well established for versioned file-archival. Though the advantages of using file deltas to transmit changes over a network are clear, specifying and widely deploying such a system efficient enough to justify itself is not as easy as it seems. There are a number of issues to overcome. First, the execution cost of computing and compressing deltas can be prohibitive--a site administrator might rather let everyone...