Results 1  10
of
25
Data Compression
 ACM Computing Surveys
, 1987
"... This paper surveys a variety of data compression methods spanning almost forty years of research, from the work of Shannon, Fano and Huffman in the late 40's to a technique developed in 1986. The aim of data compression is to reduce redundancy in stored or communicated data, thus increasing effectiv ..."
Abstract

Cited by 87 (3 self)
 Add to MetaCart
This paper surveys a variety of data compression methods spanning almost forty years of research, from the work of Shannon, Fano and Huffman in the late 40's to a technique developed in 1986. The aim of data compression is to reduce redundancy in stored or communicated data, thus increasing effective data density. Data compression has important application in the areas of file storage and distributed systems. Concepts from information theory, as they relate to the goals and evaluation of data compression methods, are discussed briefly. A framework for evaluation and comparison of methods is constructed and applied to the algorithms presented. Comparisons of both theoretical and empirical natures are reported and possibilities for future research are suggested. INTRODUCTION Data compression is often referred to as coding, where coding is a very general term encompassing any special representation of data which satisfies a given need. Information theory is defined to be the study of eff...
A Chernoff Bound For Random Walks On Expander Graphs
 SIAM J. Comput
, 1998
"... . We consider a finite random walk on a weighted graph G; we show that the fraction of time spent in a set of vertices A converges to the stationary probability #(A) with error probability exp ..."
Abstract

Cited by 80 (0 self)
 Add to MetaCart
.<F3.827e+05> We consider a finite random walk on a weighted graph<F3.539e+05><F3.827e+05> G; we show that the fraction of time spent in a set of vertices<F3.539e+05> A<F3.827e+05> converges to the stationary probability<F3.539e+05><F3.827e+05><F3.539e+05><F3.827e+05> #(A) with error probability exponentially small in the length of the random walk and the square of the size of the deviation from<F3.539e+05><F3.827e+05><F3.539e+05><F3.827e+05> #(A). The exponential bound is in terms of the expansion of<F3.539e+05> G<F3.827e+05> and improves previous results of [D. Aldous,<F3.405e+05> Probab. Engrg. Inform.<F3.827e+05> Sci., 1 (1987), pp. 3346], [L. Lovasz and M. Simonovits,<F3.405e+05> Random Structures<F3.827e+05> Algorithms, 4 (1993), pp. 359412], [M. Ajtai, J. Komlos, and E. Szemeredi,<F3.405e+05> Deterministic simulation of<F3.827e+05> logspace, in Proc. 19th ACM Symp. on Theory of Computing, 1987]. We show that taking the sample average from one trajectory gives a more e#cien...
Covering numbers for support vector machines
 IEEE Trans. Inform. Theory
, 2002
"... Abstract—Support vector (SV) machines are linear classifiers that use the maximum margin hyperplane in a feature space defined by a kernel function. Until recently, the only bounds on the generalization performance of SV machines (within Valiant’s probably approximately correct framework) took no ac ..."
Abstract

Cited by 19 (6 self)
 Add to MetaCart
Abstract—Support vector (SV) machines are linear classifiers that use the maximum margin hyperplane in a feature space defined by a kernel function. Until recently, the only bounds on the generalization performance of SV machines (within Valiant’s probably approximately correct framework) took no account of the kernel used except in its effect on the margin and radius. More recently, it has been shown that one can bound the relevant covering numbers using tools from functional analysis. In this paper, we show that the resulting bound can be greatly simplified. The new bound involves the eigenvalues of the integral operator induced by the kernel. It shows that the effective dimension depends on the rate of decay of these eigenvalues. We present an explicit calculation of covering numbers for an SV machine using a Gaussian kernel, which is significantly better than that implied by previous results. Index Terms—Covering numbers, entropy numbers, kernel machines, statistical learning theory, support vector (SV) machines. I.
An algorithm for finding the distribution of maximal entropy
 J. Comput. Phys
, 1979
"... An algorithm for determining the distribution of maximal entropy subject to constraints is presented. The method provides an alternative to the conventional procedure which requires the numerical solution of a set of implicit nonlinear equations for the Lagrange multipliers. Here they are determined ..."
Abstract

Cited by 15 (0 self)
 Add to MetaCart
An algorithm for determining the distribution of maximal entropy subject to constraints is presented. The method provides an alternative to the conventional procedure which requires the numerical solution of a set of implicit nonlinear equations for the Lagrange multipliers. Here they are determined by seeking a minimum of a concave function, a procedure which readily lends itself to computational work. The program also incorporates two preliminary stages. The first verifies that the constraints are linearly independent and the second checks that a feasible solution exists. 1. TNTR~DUCTI~N In applications of probability theory to the physical sciences [l] one is often faced with the problem of determining a distribution consistent with a given set of average values. For n distinct states one thus seeks a vector x (components xi, xi> 0, i = l,..., n), characterized by $I A,ixi = h, , r = I,..., m. (2) Here Eq. (1) is the normalization condition and (2) defines b, as the average value of the property A, , whose magnitude in the ith state is A, $. Equations (1) and (2) represent 111 + 1 constraints on the vector x and hence, if m < n 1, do not suffice to provide a unique characterization. The principle of maximal entropy [l] provides that when IIT < n 1, the probability assignment be made by the additional condition that the entropy, S[x] (or missing information [l, 21) of the distribution, S[x] = i xi In xi, i=l
Entropy Numbers, Operators and Support Vector Kernels
 IEEE TRANSACTIONS ON INFORMATION THEORY
, 1998
"... We derive new bounds for the generalization error of feature space machines, such as support vector machines and related regularization networks by obtaining new bounds on their covering numbers. The proofs are based on a viewpoint that is apparently novel in the field of statistical learning theory ..."
Abstract

Cited by 11 (3 self)
 Add to MetaCart
We derive new bounds for the generalization error of feature space machines, such as support vector machines and related regularization networks by obtaining new bounds on their covering numbers. The proofs are based on a viewpoint that is apparently novel in the field of statistical learning theory. The hypothesis class is described in terms of a linear operator mapping from a possibly infinite dimensional unit ball in feature space into a finite dimensional space. The covering numbers of the class are then determined via the entropy numbers of the operator. These numbers, which characterize the degree of compactness of the operator, can be bounded in terms of the eigenvalues of an integral operator induced by the kernel function used by the machine. As a consequence we are able to theoretically explain the effect of the choice of kernel functions on the generalization performance of support vector machines.
Complexity in Manufacturing Systems  Part 1: Analysis of Static Complexity
 IIE Transactions
, 1998
"... This paper studies static complexity in manufacturing systems. We enumerate factors influencing static complexity, and define a static complexity measure in terms of the processing requirements of parts to be produced and machine capabilities. The measure suggested for static complexity in manufa ..."
Abstract

Cited by 10 (0 self)
 Add to MetaCart
This paper studies static complexity in manufacturing systems. We enumerate factors influencing static complexity, and define a static complexity measure in terms of the processing requirements of parts to be produced and machine capabilities. The measure suggested for static complexity in manufacturing systems needs only the information available from production orders and process plans. The variation in static complexity is studied with respect to part similarity, system size, and product design changes. Finally, we present relationships between the static complexity measure and system performance. 1 Introduction Manufacturing systems are often described as being complex [Pritsker, 1990, Lin, 1993]. The dynamic nature of the manufacturing environment greatly increases the number of decisions that need to be made and system integration makes it difficult to predict the effect of a decision on future system performance. In fact, Upton [Upton, 1988] observes that many integrate...
DNA Sequence Classification via an Expectation Maximization Algorithm and Neural Networks: A Case Study
 IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews
, 2001
"... This paper presents new techniques for biosequence classification, with a focus on recognizing E. Coli promoters in DNA. Specifically, given an unlabeled DNA sequence S, we want to determine whether or not S is an E. Coli promoter. We use an expectationmaximization (EM) algorithm to locate the35 a ..."
Abstract

Cited by 9 (1 self)
 Add to MetaCart
This paper presents new techniques for biosequence classification, with a focus on recognizing E. Coli promoters in DNA. Specifically, given an unlabeled DNA sequence S, we want to determine whether or not S is an E. Coli promoter. We use an expectationmaximization (EM) algorithm to locate the35 and10 binding sites in an E. Coli promoter sequence. The EM algorithm differs from previously published EM algorithms in that, instead of assuming a uniform distribution for the lengths of the spacer between the35 binding site and the10 binding site as well as the spacer between the10 binding site and the transcriptional start site, our algorithm deduces the probability distribution for these lengths. Based on the located binding sites, we select features in each E. Coli promoter sequence according to their information contents and represent the features using an orthogonal encoding method. We then feed the features to a neural network for promoter recognition. Empirical studies show that the proposed approach achieves good performance on different datasets.
Information Theoretic Determination of Minimax Rates of Convergence
 Annals of Statistics
, 1995
"... In this paper, we present some general results determining minimax bounds on statistical risk for density estimation based on certain informationtheoretic considerations. These bounds depend only on metric entropy conditions and are used to identify the minimax rates of convergence. ..."
Abstract

Cited by 7 (1 self)
 Add to MetaCart
In this paper, we present some general results determining minimax bounds on statistical risk for density estimation based on certain informationtheoretic considerations. These bounds depend only on metric entropy conditions and are used to identify the minimax rates of convergence.
Non prefixfree codes for constrained sequences
 in International Symposium on Information Theory, 2005. ISIT 2005, IEEE
"... Abstract — In this paper we consider the use of variable length non prefixfree codes for coding constrained sequences of symbols. We suppose to have a Markov source where some state transitions are impossible, i.e. the stochastic matrix associated with the Markov chain has some null entries. We sho ..."
Abstract

Cited by 6 (0 self)
 Add to MetaCart
Abstract — In this paper we consider the use of variable length non prefixfree codes for coding constrained sequences of symbols. We suppose to have a Markov source where some state transitions are impossible, i.e. the stochastic matrix associated with the Markov chain has some null entries. We show that classic Kraft inequality is not a necessary condition, in general, for unique decodability under the above hypothesis and we propose a relaxed necessary inequality condition. This allows, in some cases, the use of non prefixfree codes that can give very good performance, both in terms of compression and computational efficiency. Some considerations are made on the relation between the proposed approach and other existing coding paradigms. I.
Wider Still And Wider...  Resetting The Bounds Of Logic
, 1997
"... Modern logic is often defined in terms of specific formal languages, rules, and calculi. Such architectural decisions about a field form a pervasive implicit definition which determines professional practice  through the structure of textbooks, as well as the research agenda that determines 'inter ..."
Abstract

Cited by 5 (1 self)
 Add to MetaCart
Modern logic is often defined in terms of specific formal languages, rules, and calculi. Such architectural decisions about a field form a pervasive implicit definition which determines professional practice  through the structure of textbooks, as well as the research agenda that determines 'interest', and hence acceptance and academic status. Such a practice may come to contain a lot of historical accident, or force of habit. Therefore, it seems worth thinking about the defining agenda of a field once in a while. In this brief essay, we explore alternative views of logic, locating the nature of the field in more abstract themes, concerns and attitudes. The new definition does not remove the need for the old agenda, but we advocate a shift in emphasis, toward greater generality and range of application. The outcome is a conception of logic as a broad methodological stance, looking for invariants in (information) structures and processes. to appear in A. Varzi, ed. "The European Revie...