Abstract

A&r&-By defining mutual information as a maximum over an appropriate space, channel capacities can be defined as double maxima and rate-distortion functions as double minima. This approach yields valuable new insights regarding the computation of channel capacities and rate-distortion functions. In particular, it suggests a simple algo-rithm for computing channel capacity that consists of a mapping from the set of channel input probability vectors into itself such that the sequence of probability vectors generated by successive applications of the mapping converges to the vector that achieves the capacity of the given channel. Analogous algorithms then are provided for computing ram-distortion functions and constrained channel capacities. The algo-rithms apply both to discrete and to continuous alphabet channels or sources. In addition, a formalization of the theory of channel capacity in the presence of constraints is included. Among the examples is the calculation of close upper and lower bounds to the rate-distortion function of a binary symmetric Markov source. C

Citations

283	A mathematical theory of communication,” Bell Syst - Shannon - 1948
173	Coding theorems for a discrete source with fidelity criterion - Shannon - 1960
8	Coding for discrete information source with a distortion measure - Goblick - 1962
2	Information rates of autoregressive sources - Gray - 1969
1	Computation of information measures - IT-17 - 1971