this paper. Let us place it within the neural network perspective, and particularly that of learning. The area of neural networks has greatly benefited from its unique position at the crossroads of several diverse scientific and engineering disciplines including statistics and probability theory, physics, biology, control and signal processing, information theory, complexity theory, and psychology (see [45]). Neural networks have provided a fertile soil for the infusion (and occasionally confusion) of ideas, as well as a meeting ground for comparing viewpoints, sharing tools, and renovating approaches. It is within the ill-defined boundaries of the field of neural networks that researchers in traditionally distant fields have come to the realization that they have been attacking fundamentally similar optimization problems.

We determine analytic expressions for the performance of some low-complexity combined source-channel coding systems. The main tool used is the Hadamard transform. In particular, we obtain formulas for the average distortion of binary lattice vector quantization with affine index assignments, linear block channel coding, and a binary-symmetric channel. The distortion formulas are specialized to nonredundant channel codes for a binary-symmetric channel, and then extended to affine index assignments on a binary-asymmetric channel. Various structured index assignments are compared. Our analytic formulas provide a computationally efficient method for determining the performance of various coding schemes. One interesting result shown is that for a uniform source and uniform quantizer, the Natural Binary Code is never optimal for a nonsymmetric channel, even though it is known to be optimal for a symmetric channel. Index Terms--- Index assignment, lattices, linear error-correcting codes, sou...

We propose three new design algorithms for jointly optimizing source and channel codes. Our optimality criterion is to minimize the average end-to-end distortion. For a given channel SNR and transmission rate, our joint source and channel code designs achieve an optimal allocation of bits between the source and channel coders. Our three techniques include a sourceoptimized channel code, a channel-optimized source code, and an iterative descent technique combining the design strategies of the other two codes. The joint designs use channel-optimized vector quantization (COVQ) for the source code and rate-compatible punctured convolutional (RCPC) coding for the channel code. The optimal bit allocation reduces distortion by up to 6 dB over suboptimal allocations and by up to 4 dB relative to standard COVQ for the source data set considered. We find that all three code designs have roughly the same performance when their bit allocations are optimized. This result follows from the fact that at the optimal bit allocation the channel code removes most of the channel errors, in which case the three design techniques are roughly equivalent. We also compare the robustness of the three techniques to channel mismatch. We conclude the paper by relaxing the fixed transmission rate constraint and jointly optimizing the transmission rate, source code, and channel code. Index Terms---Joint source/channel coding, optimal bit allocation, RCPC channel code, vector quantization. I.

We propose algorithms to design channel-optimized vector quantizers in the presence of channel mismatch. We consider two cases: (i) no information about the statistics of the channel bit error rate is available and (ii) the probability density function of the channel bit error rate is known. We also consider the use of an estimate of the channel signalto -noise ratio to improve performance. Simulation results demonstrate the advantages of new design algorithms. 1. INTRODUCTION Source coding applications which involve transmission over noisy channels have been the main motivation for studying the sensitivity of a vector quantizer (VQ) to channel noise. These studies have led to the development of techniques for making a VQ robust with respect to channel noise, either by an appropriate binary codeword assignment [1, 2] or by a complete redesign of the VQ partition and codebook, resulting in the so-called channel-optimized vector quantizer (COVQ) [3, 4]. Previous studies on the subject h...

This paper presents a general framework for combined source-channel coding within the context of subband coding. The unequal importance of subbands in reconstruction of the source is exploited by an appropriate allocation of source and channel coding rates for the coding and transmission of subbands over a noisy channel. For each subband, the source coding rate as well as the level of protection (quantified by the channel coding rate) are jointly chosen to minimize the total end-to-end mean-squared distortion suffered by the source. This allocation of source and channel coding rates is posed as a constrained optimization problem, and solved using a generalized bit allocation algorithm. The optimal choice of source and channel coding rates depends on the state of the physical channel. These results are extended to transmission over fading channels using a finite state model, where every state corresponds to an AWGN channel. A coding strategy is also developed that minimizes the average ...

Abstract — An approach to optimal soft decoding for vector quantization (VQ) over a code-division multiple-access (CDMA) channel is presented. The decoder of the system is soft in the sense that the unquantized outputs of the matched filters are utilized directly for decoding (no decisions are taken), and optimal according to the minimum mean-squared error (MMSE) criterion. The derived decoder utilizes a priori source information and knowledge of the channel characteristics to combat channel noise and multiuser interference in an optimal fashion. Hadamard transform representations for the user VQ’s are employed in the derivation and for the implementation of the decoder. The advantages of this approach are emphasized. Suboptimal versions of the optimal decoder are also considered. Simulations show the soft decoders to outperform decoding based on maximum-likelihood (ML) multiuser detection. Furthermore, the suboptimal versions are demonstrated to perform close to the optimal, at a significantly lower complexity in the number of users. The introduced decoders are, moreover, shown to exhibit near–far resistance. Simulations also demonstrate that combined source–channel encoding, with joint source–channel and multiuser decoding, can significantly outperform a tandem source–channel coding scheme employing multiuser detection plus table lookup source decoding. Index Terms—Code-division multiple access, combined source and channel coding, joint optimization, multiuser detection, multiuser systems, soft decoding, vector quantization. I.

I hereby declare that I am the sole author of this thesis. I authorize the University of Waterloo to lend this thesis to other institutions or individuals for the purpose of scholarly research. I further authorize the University of Waterloo to reproduce this thesis by pho-tocopying or by other means, in total or in part, at the request of other institutions or individuals for the purpose of scholarly research. ii The University of Waterloo requires the signatures of all persons using or pho-tocopying this thesis. Please sign below, and give address and date. iii A fundamental problem in telecommunications is the reliable transmission of a source over a noisy channel. As an important result of the Shannon’s celebrated paper [1], the problem can be theoretically separated, without loss of optimality, into two parts: source coding and channel coding. However, in practise, due to

: An approach to joint equalization and decoding for vector quantization over a Gaussian channel with intersymbol interference is presented. The decoder is based on a Hadamard transform representation of the vector quantizer. This gives the decoder a structure that allows the decoding to be based on estimates of the transmitted bits in an efficient manner. The decoder is soft in the sense that soft channel information is used and no decisions are involved in the decoding. Simulations demonstrate that the soft decoder can outperform a scheme based on Viterbi equalization plus table look-up decoding. I. INTRODUCTION Most channels corrupt the transmitted signal in some way, for example with thermal or impulsive noise, and fading. One of the major obstacles for high speed transmission is time dispersion. In telephone lines, for example, time dispersion results from the frequency dependent channel characteristics. In bandwidth-efficient digital communication systems based on pulse amplitu...

The combination of a source coder and a digital modulator is in this article viewed as an analog-to-analog converter with signal compression ability. A reference channel with defined available bandwidth, total received power, and noise statistics, can transmit one analog source signal. Compression is here defined as the number of such source signals that the digital system can transmit over the reference channel, and reconstruct with the same fidelity as the transmitted analog source signal. An image transmission example is provided where, for a given fidelity on a reconstructed image, the obtainable compression ratio is found for a digital compression system based on subband coding and pulse amplitude modulation. 1. INTRODUCTION Digital compression of analog signals is used to facilitate signal transmission or storage. All transmission and storage media are, however, in nature analog and the compressed digital signals must therefore be adapted to the media through some form of digita...

We propose a channel-matched hierarchical table-lookup vector quantizer (CM-HTVQ) which provides some robustness against the channel noise. We use a finite-state channel to model slowly fading channels and propose an adaptive coding scheme to transmit a source over wireless channels. The performance of CM-HTVQ is in general slightly inferior to that of channel-optimized vector quantizer (COVQ) (the performances coincide at some cases); however, the encoder complexity of CM-HTVQ is much less than the encoder complexity of COVQ. 1. INTRODUCTION Vector quantization is a powerful tool for source coding which has been used in many speech and image coding systems [1]. The encoder of a vector quantizer (VQ) is usually implemented by computing the distortion between the input vector and each codevector in the codebook and finding the codevector which results in minimum distortion. The decoder however is a simple table lookup. In [2], Chang et al. have proposed a hierarchical table-lookup vec...