The problem of error control and concealment in video communication is becoming increasingly important because of the growing interest in video delivery over unreliable channels such as wireless networks and the Internet. This paper reviews the techniques that have been developed for error control and concealment in the past ten to fifteen years. These techniques are described in three categories according to the roles that the encoder and decoder play in the underlying approaches. Forward error concealment includes methods that add redundancy at the source end to enhance error resilience of the coded bit streams. Error concealment by postprocessing refers to operations at the decoder to recover the damaged areas based on characteristics of image and video signals. Finally, interactive error concealment covers techniques that are dependent on a dialog between the source and destination. Both current research activities and practice in international standards are covered.

Real-time audio and video data streamed over unreliable IP networks, such as the Internet, may encounter losses due to dropped packets or late arrivals. This paper reviews error-concealment schemes developed for streaming realtime audio and video data over the Internet. Based on their interactions with (video or audio) source coders, we classify existing techniques into source coder-independent schemes that treat underlying source coders as black boxes, and source coder-dependent schemes that exploit coder-specific characteristics to perform reconstruction. Last, we identify possible future research directions. 1. Introduction Increases in bandwidth and computational speed lead to growing interests in real-time audio and video transmissions over the Internet. In the Internet, packets carrying real-time data may be dropped or arrive too late to be useful because the Internet is a packet-switched, best-effort delivery service, with no guarantee on the quality of service (QoS). Traditi...

In this paper, we focus on the total-system-energy mininfization of a wireless image transntission system including both digital and analog components. Traditionally, digital power consumption has been ignored in system design, since transnfit power has been the most significant component. However, as we move to an era of pico-cell environments and as more complex signal processing algorithms are being used at higher data rates, the digital power consumption of these systems becomes an issue. We present an energy-optinfized image transnfission system for indoor wireless applications which exploits the variabilities in the image data and the wireless multipath channel by employing dynamic algorithm transformations and joint source-channel coding. The variability in the image data is characterized by the rate-distortion curve, and the variability in the channel characteristics is characterized by the path-loss and impulse response of the channel. The system hardware configuration space is characterized by the error-correction capability of the channel encoder/decoder, number of powered-up fingers in the RAKE receiver, and transmit power of the power amplifier. An optimization algorithm is utilized to obtain energy-optimal configurations subject to end-to-end performance constraints. The proposed design is tested over QCIF images, IMT-2000 channels and 0.18/m, 2.5V CMOS technology parameters. Simulation results over various images, various distances, two different channels, and two different rates show that the average energy savings in utilizing a total-system-energy mininfization over a fixed system (designed for the worst image, the worst channel and the maximum distance) are 53.6% and 67.3%, respectively, for short-range (under 20m) and long-range (o...

We propose a progressively reliable transport protocol for delivery of delay-sensitive multimedia over Internet connections with wireless access links. The protocol, termed "Leaky" ARQ, initially permits corrupt packets to be leaked to the receiving application and then uses retransmissions to progressively refine the quality of subsequent packet versions. A Web server would employ Leaky ARQ to quickly deliver a possibly corrupt first version of an image over a noisy bandlimited wireless link for immediate display by a Web browser. Later, Leaky ARQ's retransmissions would enable the browser to eventually display a cleaner image. Forwarding and displaying corrupt error-tolerant image data: (1) lowers the perceptual delay compared to fully reliable packet delivery, and (2) can be shown to produce images with lower distortion than aggressively compressed images when the delay budget only permits weak forward error correction. Leaky ARQ supports delaying of re-transmissions so that initial packet transmissions can be expedited, and cancelling of retransmissions associated with "out-of-date" data. Leaky ARQ can be parameterized to partially retransmit audio and video. We propose to implement Leaky ARQ by modifying Type-II Hybrid/"code combining" ARQ.

This paper presents a general framework for combined source-channel coding within the context of subband coding. The unequal importance of subbands in reconstruction of the source is exploited by an appropriate allocation of source and channel coding rates for the coding and transmission of subbands over a noisy channel. For each subband, the source coding rate as well as the level of protection (quantified by the channel coding rate) are jointly chosen to minimize the total end-to-end mean-squared distortion suffered by the source. This allocation of source and channel coding rates is posed as a constrained optimization problem, and solved using a generalized bit allocation algorithm. The optimal choice of source and channel coding rates depends on the state of the physical channel. These results are extended to transmission over fading channels using a finite state model, where every state corresponds to an AWGN channel. A coding strategy is also developed that minimizes the average ...

We attack the problem of robust and efficient image compression for transmission over noisy channels. To achieve the dual goals of high compression efficiency and low sensitivity to channel noise we introduce a multimode coding framework. Multimode coders are quasi-fixed length in nature, and allow optimization of the tradeoff between the compression capability of variable-length coding and the robustness to channel errors of fixed length coding. We apply our framework to develop multimode image coding (MIC) schemes for noisy channels, based on the adaptive DCT. The robustness of the proposed MIC is further enhanced by the incorporation of a channel protection scheme suitable for the constraints on complexity and delay. To demonstrate the power of the technique we develop two specific image coding algorithms optimized for the binary symmetric channel. The first, MIC1, incorporates channel optimized quantizers and the second, MIC2, uses rate compatible punctured convolutional codes with...

A DCT-based combined source-channel coding system is proposed for transmitting grey-level images over a binary channel with additive Markov noise. It consists of a channel optimized quantization scheme that exploits the channel memory by incorporating the characteristics of the correlated noise in the quantizer design. Experimental results show that this simple system -- which employs a fixed zonal coding bit allocation technique -- provides significant improvements over traditional tandem systems, especially during bad channel conditions. The loss of optimality due to the use of the zonal coding bit allocation method is also examined. The loss is shown to be small for various images; this suggests that a reduction in complexity and bandwidth requirements can further be achieved.

Two common source-channel coding strategies, tandem coding and channel-optimized quantization, are compared on the basis of distortion vs. complexity by analyzing specific representatives of each when transmitting analog data samples across a binary symmetric channel. Transform coding is used in each strategy, and Reed-Solomon codes are added to form the tandem code. For each strategy, formulas for the mean-squared error and complexity are found and used to optimize it so as to minimize distortion subject to a complexity constraint, for data modeled as Gauss-Markov. The results of such optimizations suggest there is a complexity threshold such that when the number of available operations per sample is above this threshold, tandem coding is better, and when below the threshold, channel-optimized transform coding is better. 1. Introduction Two commonly cited strategies for communicating analog data samples across a binary symmetric channel are: (1) tandem coding, in which a qu...

A progressive method for transmission of images over a bursty noise channel is presented. It is based on discrete wavelet transform (DWT) coding and channel-optimized scalar quantization. The main advantage of the proposed system is that it exploits the channel memory and hence has superior performance over a similar scheme designed for the equivalent memoryless channel through the use of channel interleaving. In fact, the performance of the proposed system improves as the noise becomes more correlated, at a xed bit error rate. Comparisons are made with other alternatives which employ independent source and channel coding over the fully interleaved channel at various bit rates and bit error rates. It is shown that the proposed method outperforms these substantially more complex systems for the whole range of considered bit rates and for a wide range of channel conditions.

Subband-coded images can be transmitted in the Internet using either the TCP or the UDP protocol. Delivery by TCP gives superior decoding quality but with very long delays when the network is unreliable, whereas delivery by UDP has negligible delays but with degraded quality when packets are lost. Although images are delivered currently over the Internet by TCP, we study in this paper the use of UDP to deliver multi-description reconstruction-based subbandcoded images. First, in order to facilitate recovery from UDP packet losses, we propose a joint sender-receiver approach for designing optimized reconstruction-based subband transform (ORBST) in multi-description coding (MDC). Second, we carefully evaluate the delay-quality tradeoffs between the TCP delivery of SDC images and the UDP and combined TCP/UDP delivery of MDC images. Experimental results show that our proposed ORB-ST performs well in real Internet tests, and UDP and combined TCP/UDP delivery of MDC images provide a range of attractive alternatives to TCP delivery. Keywords: single-description coding (SDC), multi-description coding (MDC), real-time multimedia in the Internet, reconstruction-based subband image coding, TCP, UDP, World-Wide Web 1.