The problem of error control and concealment in video communication is becoming increasingly important because of the growing interest in video delivery over unreliable channels such as wireless networks and the Internet. This paper reviews the techniques that have been developed for error control and concealment in the past ten to fifteen years. These techniques are described in three categories according to the roles that the encoder and decoder play in the underlying approaches. Forward error concealment includes methods that add redundancy at the source end to enhance error resilience of the coded bit streams. Error concealment by postprocessing refers to operations at the decoder to recover the damaged areas based on characteristics of image and video signals. Finally, interactive error concealment covers techniques that are dependent on a dialog between the source and destination. Both current research activities and practice in international standards are covered.

Video communication over lossy packet networks suchastheInternet is hampered by limited bandwidth and packet loss. This paper presents a system for providing reliable video communication over these networks, where the system is composed of two subsystems: (1) multiple state video encoder/decoder and (2) a path diversity transmission system. Multiple state video coding combats the problem of error propagation at the decoder by coding the video into multiple independently decodable streams, each with its own prediction process and state. If one stream is lost the other streams can still be decoded to produce usable video, and furthermore, the correctly received streams provide bidirectional (previous and future) information that enables improved state recovery for the corrupted stream. This video coder is a form of multiple description coding (MDC), and its novelty lies in its use of information from the multiple streams to perform state recovery at the decoder. The path diversity transmission system explicitly sends different subsets of packets over different paths, as opposed to the default scenarios where the packets proceed along a single path, thereby enabling the end-to-end video application to effectively see an average path behavior. We refer to this as path diversity. Generally, seeing this average path behavior provides better performance than seeing the behavior of any individual random path. For example, the probability that all of the multiple paths are simultaneously congested is much less than the probability that a single path is congested. The resulting path diversityprovides the multiple state video decoder with an appropriate virtual channel to assist in recovering from lost packets, and can also simplify system design, e.g. FEC design. Weproposetwoarch...

this paper, we discuss such last-line-of-defense 0018--9219/99$10.00 1999 IEEE PROCEEDINGS OF THE IEEE, VOL. 87, NO. 10, OCTOBER 1999 1707 techniques that can be used to make low bit-rate video coders error resilient. We concentrate on techniques that use acknowledgment information provided by a feedback channel

Abstract—This paper addresses the resource allocation problem for multiple media streaming over the Internet. First, we present an end-to-end transport architecture for multimedia streaming over the Internet. Second, we propose a new multimedia streaming TCP-friendly protocol (MSTFP), which combines forward estimation of network conditions with information feedback control to optimally track the network conditions. Third, we propose a novel resource allocation scheme to adapt media rate to the estimated network bandwidth using each media’s rate-distortion function under various network conditions. By dynamically allocating resources according to network status and media characteristics, we improve the end-to-end quality of services (QoS). Simulation results demonstrate the effectiveness of our proposed schemes. Index Terms—Flow control, multimedia streaming, QoS, rate control, resource allocation, TCP-friendly.

Multimedia transmission has to handle a variety of compressed and uncompressed source

Real-time audio and video data streamed over unreliable IP networks, such as the Internet, may encounter losses due to dropped packets or late arrivals. This paper reviews error-concealment schemes developed for streaming realtime audio and video data over the Internet. Based on their interactions with (video or audio) source coders, we classify existing techniques into source coder-independent schemes that treat underlying source coders as black boxes, and source coder-dependent schemes that exploit coder-specific characteristics to perform reconstruction. Last, we identify possible future research directions. 1. Introduction Increases in bandwidth and computational speed lead to growing interests in real-time audio and video transmissions over the Internet. In the Internet, packets carrying real-time data may be dropped or arrive too late to be useful because the Internet is a packet-switched, best-effort delivery service, with no guarantee on the quality of service (QoS). Traditi...

Video compression enables a number of applications by reducing the required bit rate needed to represent a video sequence, however the compressed video is much more susceptible to errors such as bit errors or packet loss. Conventional video compression standards employ an architecture which we refer to as single-state systems since they have a prediction loop with a single state (e.g. the previous decoded frame) which if lost or corrupted can lead to the loss or severe degradation of all subsequent frames until the state is reinitialized (the prediction is refreshed). We combat this problem of incorrect state and error propagation at the decoder by coding the video into multiple independently decodable streams, each with its own prediction process and state, such that if one stream is lost the other streams can still be used to produce usable video. The correctly received streams provide improved error concealment and, more importantly, enable faster state recovery for the lost stream. This approach is conceptually similar to multiple description coding, e.g. [1], however it differs in the representation used for each description as well as its use of state recovery.

Progressive image transmission is difficult in the presence of a noisy channel, mainly due to the propagation of errors during the decoding of a progressive bitstream. Excellent results for this problem are made possible through combined source-channel coding -- a method that matches the channel code to the source operational rate-distortion as well as channel conditions. This paper focuses on the key component of combined source-channel coding: rate allocation. We develop a parametric methodology for rate allocation in progressive source-channel coding. The key to this technique is an empirical model of decoded bit error rate (BER) as a function of the channel code rate. We investigate several scenarios. In the case of the memoryless channel, we present closed form expressions. For the fading channel and channels with feedback, where closed form results are elusive, our analysis leads to low-complexity algorithms. The results presented in this paper are applicable to any progressive source code, and any family of channel codes.

In this paper we review error resilience techniques for real-time video transport over unreliable networks. Topics covered include an introduction to today's protocol and network environments and their characteristics, encoder error resilience tools, decoder error concealment techniques, as well as techniques that require cooperation between encoder, decoder and the network. We provide a review of general principles of these techniques as well as specific implementations adopted by the H.263 and MPEG-4 video coding standards. The majority of the paper is devoted to the techniques developed for block-based hybrid coders using motion-compensated prediction and transform coding. A separate section covers error resilience techniques for shape coding in MPEG-4. I. Introduction A. Error Resilience in Video Communications: Importance and Approach A video communications system typically involves five steps, as shown in Figure 1. The video is first compressed by a video encoder to reduce th...

Resynchronizing variable-length codes (RVLCs) for large alphabets are designed by first creating resynchronizing Huffman codes and then adding an extended synchronizing codeword, and the RVLCs are applied to both JPEG and wavelet-based image compression. The RVLCs demonstrate the desired resynchronization properties, both at a symbol level and structurally so that decoded data can be correctly placed within an image following errors. The encoded images, when subject to both structural and statistical error detection and concealment, can tolerate BERs of up to and are very tolerant of burst errors. The RVLC-JPEG images have negligible overhead at visually lossless bit rates, while the RVLC-wavelet overhead can be adjusted based on the desired tolerance to burst errors and typically ranges from 7-18%. The tolerance to both bit and burst errors demonstrates that images coded with such RVLCs can be transmitted over imperfect channels suffering bit errors or packet losses without channel co...