A factor graph is a bipartite graph that expresses how a "global" function of many variables factors into a product of "local" functions. Factor graphs subsume many other graphical models including Bayesian networks, Markov random fields, and Tanner graphs. Following one simple computational rule, the sum-product algorithm operates in factor graphs to compute---either exactly or approximately---various marginal functions by distributed message-passing in the graph. A wide variety of algorithms developed in artificial intelligence, signal processing, and digital communications can be derived as specific instances of the sum-product algorithm, including the forward/backward algorithm, the Viterbi algorithm, the iterative "turbo" decoding algorithm, Pearl's belief propagation algorithm for Bayesian networks, the Kalman filter, and certain fast Fourier transform algorithms.

In this paper we will introduce an ensemble of codes called irregular repeat-accumulate (IRA) codes. IRA codes are a generalization of the repeat-accumulate codes introduced in [1], and as such have a natural linear time encoding algorithm. We shall prove that on the binary erasure channel, IRA codes can be decoded reliably in linear time, using iterater] sum-product decoding,a# ra#SJ a#SJ8T a#SJ8 close tocha#T36 ca pa#J464 Asimila# resulta#u ea#S to be true on the AWGN channel, although we have no proof of this. We illustrate our results with numerical and experimenta# examples.

A large variety of algorithms in coding, signal processing, and artificial intelligence may be viewed as instances of the summary-product algorithm (or belief/probability

Abstract—Extrinsic information transfer (EXIT) charts are a tool for predicting the convergence behavior of iterative processors for a variety of communication problems. A model is introduced that applies to decoding problems, including the iterative decoding of parallel concatenated (turbo) codes, serially concatenated codes, low-density parity-check (LDPC) codes, and repeat–accumulate (RA) codes. EXIT functions are defined using the model, and several properties of such functions are proved for erasure channels. One property expresses the area under an EXIT function in terms of a conditional entropy. A useful consequence of this result is that the design of capacity-approaching codes reduces to a curve-fitting problem for all the aforementioned codes. A second property relates the EXIT function of a code to its Helleseth–Kløve–Levenshtein information functions, and thereby to the support weights of its subcodes. The relation is via a refinement of information functions called split information functions, and via a refinement of support weights called split support weights. Split information functions are used to prove a third property that relates the EXIT function of a linear code to the EXIT function of its dual. Index Terms—Concatenated codes, duality, error-correction coding, iterative decoding, mutual information.

Abstract. Error-correcting codes are fundamental tools used to transmit digital information over unreliable channels. Their study goes back to the work of Hamming [Ham50] and Shannon [Sha48], who used them as the basis for the field of information theory. The problem of decoding the original information up to the full error-correcting potential of the system is often very complex, especially for modern codes that approach the theoretical limits of the communication channel. In this thesis we investigate the application of linear programming (LP) relaxation to the problem of decoding an error-correcting code. Linear programming relaxation is a standard technique in approximation algorithms and operations research, and is central to the study of efficient algorithms to find good (albeit suboptimal) solutions to very difficult optimization problems. Our new “LP decoders ” have tight combinatorial characterizations of decoding success that can be used to analyze error-correcting performance. Furthermore, LP decoders have the desirable (and rare) property that whenever they output a result, it is guaranteed to be the optimal result: the most likely (ML) information sent over the

We optimize the random-like ensemble of Irregular Repeat Accumulate (IRA) codes for binary-input symmetric channels in the large blocklength limit. Our optimization technique is based on approximating the Evolution of the Densities (DE) of the messages exchanged by the Belief-Propagation (BP) message-passing decoder by a one-dimensional dynamical system. In this way, the code ensemble optimization can be solved by linear programming. We propose four such DE approximation methods, and compare the performance of the obtained code ensembles over the binary symmetric channel (BSC) and the binaryantipodal input additive white Gaussian channel (BIAWGNC). Our results clearly identify the best among the proposed methods and show that the IRA codes obtained by these methods are competitive with respect to the best-known irregular Low-Density Parity-Check codes (LDPC). In view of this and the very simple encoding structure of IRA codes, they emerge as attractive design choices.

In this paper, we review recent developments concerning the application of lowdensity parity-check (LDPC) codes to the Gilbert-Elliott (GE) channel. Firstly, we discuss the analysis of LDPC estimation-decoding in these channels using density evolution. We show that the required conditions of density evolution are satisfied in the GE channel, and that analysis demonstrates that large potential gains over the memoryless assumption. We also give results which mitigate the complexity of characterizing the GE parameter space using DE. Subsequently, we give a design tool for finding good degree sequences for irregular LDPC codes in the GE channel.

Abstract. A new, simple, approach for active steganography is proposed in this paper that can successfully resist recent blind steganalysis methods, in addition to surviving distortion constrained attacks. We present Yet Another Steganographic Scheme (YASS), a method based on embedding data in randomized locations so as to disable the selfcalibration process (such as, by cropping a few pixel rows and/or columns to estimate the cover image features) popularly used by blind steganalysis schemes. The errors induced in the embedded data due to the fact that the stego signal must be advertised in a specific format such as JPEG, are dealt with by the use of erasure and error correcting codes. For the presented JPEG steganograhic scheme, it is shown that the detection rates of recent blind steganalysis schemes are close to random guessing, thus confirming the practical applicability of the proposed technique. We also note that the presented steganography framework, of hiding in randomized locations and using a coding framework to deal with errors, is quite simple yet very generalizable. Key words: data hiding, error correcting codes, steganalysis, steganography, supervised learning. 1

We derive worst case upper bounds on the minimum distance of parallel concatenated convolutional codes, serial concatenated convolutional codes, repeat-and-accumulate codes, and various generalizations. We conclude from the bounds that in all these cases the relative minimum distance goes to zero as the block length tends to innity.

We introduce a novel algorithm for decoding turbo-like codes based on linear programming. We prove that for the case of Repeat-Accumulate (RA) codes, under the binary symmetric channel with a certain constant threshold bound on the noise, the error probability of our algorithm is bounded by an inverse polynomial in the code length. Our linear program (LP) minimizes the distance between the received bits and binary variables representing the code bits. Our LP is based on a representation of the code where code words are paths through a graph. Consequently, the LP bears a strong resemblance to the min-cost flow LP. The error bounds are based on an analysis of the probability, over the random noise of the channel, that the optimum solution to the LP is the path corresponding to the original transmitted code word.