Results 1 
9 of
9
Fast and efficient compression of floatingpoint data
 IEEE Transactions on Visualization and Computer Graphics
, 2006
"... Abstract—Large scale scientific simulation codes typically run on a cluster of CPUs that write/read time steps to/from a single file system. As data sets are constantly growing in size, this increasingly leads to I/O bottlenecks. When the rate at which data is produced exceeds the available I/O band ..."
Abstract

Cited by 39 (5 self)
 Add to MetaCart
(Show Context)
Abstract—Large scale scientific simulation codes typically run on a cluster of CPUs that write/read time steps to/from a single file system. As data sets are constantly growing in size, this increasingly leads to I/O bottlenecks. When the rate at which data is produced exceeds the available I/O bandwidth, the simulation stalls and the CPUs are idle. Data compression can alleviate this problem by using some CPU cycles to reduce the amount of data needed to be transfered. Most compression schemes, however, are designed to operate offline and seek to maximize compression, not throughput. Furthermore, they often require quantizing floatingpoint values onto a uniform integer grid, which disqualifies their use in applications where exact values must be retained. We propose a simple scheme for lossless, online compression of floatingpoint data that transparently integrates into the I/O of many applications. A plugin scheme for datadependent prediction makes our scheme applicable to a wide variety of data used in visualization, such as unstructured meshes, point sets, images, and voxel grids. We achieve stateoftheart compression rates and speeds, the latter in part due to an improved entropy coder. We demonstrate that this significantly accelerates I/O throughput in real simulation runs. Unlike previous schemes, our method also adapts well to variableprecision floatingpoint and integer data. Index Terms—High throughput, lossless compression, file compaction for I/O efficiency, fast entropy coding, range coder, predictive coding, large scale simulation and visualization. 1
Lossless Geometry Compression for SteadyState and
 In Proc. Eurographics/IEEE Symposium on Visualization (EuroVis ’06
, 2005
"... In this paper we investigate the problem of lossless geometry compression of irregulargrid volume data represented as a tetrahedral mesh. We propose a novel lossless compression technique that effectively predicts, models, and encodes geometry data for both steadystate (i.e., with only a single ..."
Abstract

Cited by 3 (2 self)
 Add to MetaCart
In this paper we investigate the problem of lossless geometry compression of irregulargrid volume data represented as a tetrahedral mesh. We propose a novel lossless compression technique that effectively predicts, models, and encodes geometry data for both steadystate (i.e., with only a single time step) and timevarying datasets.
Lossless Compression of PointBased 3D Models
"... this paper, we present a singleresolution technique for lossless compression of pointbased 3D models. An initial quantization step is not needed and we can achieve a truly lossless compression. The scheme can compress geometry information as well as attributes associated with the points. We employ ..."
Abstract
 Add to MetaCart
this paper, we present a singleresolution technique for lossless compression of pointbased 3D models. An initial quantization step is not needed and we can achieve a truly lossless compression. The scheme can compress geometry information as well as attributes associated with the points. We employ a threestage pipeline that uses several ideas including kdtreelike partitioning, minimumspanningtree modeling, and a twolayer modified Huffman coding technique based on an optimal alphabet partitioning approach using a greedy heuristic. We show that the proposed technique achieves excellent lossless compression results
Abstract Fast and Efficient Compression of FloatingPoint Data
"... Large scale scientific simulation codes typically run on a cluster of CPUs that write/read time steps to/from a single file system. As data sets are constantly growing in size, this increasingly leads to I/O bottlenecks. When the rate at which data is produced exceeds the available I/O bandwidth, th ..."
Abstract
 Add to MetaCart
(Show Context)
Large scale scientific simulation codes typically run on a cluster of CPUs that write/read time steps to/from a single file system. As data sets are constantly growing in size, this increasingly leads to I/O bottlenecks. When the rate at which data is produced exceeds the available I/O bandwidth, the simulation stalls and the CPUs are idle. Data compression can alleviate this problem by using some CPU cycles to reduce the amount of data that needs to be transfered. Most compression schemes, however, are designed to operate offline and try to maximize compression, not online throughput. Furthermore, they often require quantizing floatingpoint values onto a uniform integer grid, which disqualifies their use in applications where exact values must be retained. We propose a simple and robust scheme for lossless, online compression of floatingpoint data that transparently integrates into the I/O of a large scale simulation cluster. A plugin scheme for datadependent prediction makes our scheme applicable to a wide variety of data sets used in visualization, such as unstructured meshes, point sets, images, and voxel grids. We achieve stateoftheart compression rates and compression speeds, the latter in part due to an improved entropy coder. We demonstrate that this significantly accelerates I/O throughput in real simulation runs. Unlike previous schemes, our method also adapts well to variableprecision floatingpoint and integer data.
Information Processing Letters 97 (2006) 133137
 Information Processing Letters
"... this paper we measure fidelity using relative entropy because, in the asymmetric communication example above, the relative entropy is roughly how many more bits we expect the client to send with Email address: travis@cs.toronto.edu (T. Gagie) ..."
Abstract
 Add to MetaCart
this paper we measure fidelity using relative entropy because, in the asymmetric communication example above, the relative entropy is roughly how many more bits we expect the client to send with Email address: travis@cs.toronto.edu (T. Gagie)
Lossless Geometry Compression for FloatingPoint Data in SteadyState and TimeVarying Fields over Irregular Grids
"... Abstract — In this paper we investigate the problem of lossless geometry compression of irregulargrid volume data represented as a tetrahedral mesh. We propose a novel lossless compression technique that effectively predicts, models, and encodes geometry data for both steadystate (i.e., with only ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract — In this paper we investigate the problem of lossless geometry compression of irregulargrid volume data represented as a tetrahedral mesh. We propose a novel lossless compression technique that effectively predicts, models, and encodes geometry data for both steadystate (i.e., with only a single time step) and timevarying datasets. Our geometry coder applies to floatingpoint data without requiring an initial quantization step and is truly lossless. However, the technique works equally well even if quantization is performed. Moreover, it does not need any connectivity information, and can be easily integrated with a class of the best existing connectivity compression techniques for tetrahedral meshes with a small amount of overhead information. We present experimental results which show that our technique achieves superior compression ratios. Index Terms — graphics compression, lossless geometry compression, irregular grids, steadystate and timevarying fields. 1.
Lossless Geometry Compression for SteadyState and TimeVarying Irregular Grids †
"... In this paper we investigate the problem of lossless geometry compression of irregulargrid volume data represented as a tetrahedral mesh. We propose a novel lossless compression technique that effectively predicts, models, and encodes geometry data for both steadystate (i.e., with only a single ti ..."
Abstract
 Add to MetaCart
(Show Context)
In this paper we investigate the problem of lossless geometry compression of irregulargrid volume data represented as a tetrahedral mesh. We propose a novel lossless compression technique that effectively predicts, models, and encodes geometry data for both steadystate (i.e., with only a single time step) and timevarying datasets. Our geometry coder is truly lossless and also does not need any connectivity information. Moreover, it can be easily integrated with a class of the best existing connectivity compression techniques for tetrahedral meshes with a small amount of overhead information. We present experimental results which show that our technique achieves superior compression ratios, with reasonable encoding times and fast (linear) decoding times. 1.
On the Reduction of Entropy Coding Complexity via Symbol Grouping: I – Redundancy Analysis and Optimal Alphabet Partition Amir
, 2004
"... data compression, symbol grouping, dynamic programming, Monge matrices We analyze the technique for reducing the complexity of entropy coding that consists in the a priori grouping of the source alphabet symbols, and in the decomposition of the coding process in two stages: first coding the number o ..."
Abstract
 Add to MetaCart
(Show Context)
data compression, symbol grouping, dynamic programming, Monge matrices We analyze the technique for reducing the complexity of entropy coding that consists in the a priori grouping of the source alphabet symbols, and in the decomposition of the coding process in two stages: first coding the number of the symbol's group with a more complex method, followed by coding the symbol's rank inside its group using a less complex method, or simply using its binary representation. This technique proved to be quite effective, yielding great reductions in complexity with reasonably small losses in compression, even when the groups are designed with empiric methods. It is widely used in practice and it is an important part in standards like MPEG and JPEG. However, the theory to explain its effectiveness and optimization had not been sufficiently developed. In this work, we provide a theoretical analysis of the properties of these methods in general circumstances. Next, we study the problem of finding optimal source alphabet partitions. We demonstrate a necessary optimality condition that eliminates most of the possible solutions, and guarantees that a more constrained version of the problem, which can be solved via dynamic programming, provides the optimal solutions. In addition, we show that the data used by the dynamic programming optimization has properties similar to the Monge matrices, allowing the use of much more efficient solution methods.