The problem of error control and concealment in video communication is becoming increasingly important because of the growing interest in video delivery over unreliable channels such as wireless networks and the Internet. This paper reviews the techniques that have been developed for error control and concealment in the past ten to fifteen years. These techniques are described in three categories according to the roles that the encoder and decoder play in the underlying approaches. Forward error concealment includes methods that add redundancy at the source end to enhance error resilience of the coded bit streams. Error concealment by postprocessing refers to operations at the decoder to recover the damaged areas based on characteristics of image and video signals. Finally, interactive error concealment covers techniques that are dependent on a dialog between the source and destination. Both current research activities and practice in international standards are covered.

In this paper we address the problem of robust video transmission in error prone environments. The approach is compatible with the ITU-T video coding standard H.263. Fading situations in mobile networks are tolerated and the image quality degradation due to spatio-temporal error propagation is minimized utilizing a feedback channel between transmitter and receiver carrying acknowledgment information. In a first step, corrupted Group of Blocks (GOBs) are concealed to avoid annoying artifacts caused by decoding of an erroneous bit stream. The GOB and the corresponding frame number are reported to the transmitter via the back channel. The encoder evaluates the negative acknowledgments and reconstructs the spatial and temporal error propagation. A low complexity algorithm for real-time reconstruction of spatio-temporal error propagation is described in detail. Rapid error recovery is achieved by INTRA refreshing image regions (Macroblocks) bearing visible distortion. The feedback channel m...

The single class best effort service available in the current Internet does not provide the guarantees, typically expressed in terms of minimum bandwidth and/or maximum delay or loss, associated with real-time applications such as live video. One way to support such applications in best effort networks is to use control mechanisms that adapt the coding, transmission, reception, and decoding processes at the source and at the destination(s) depending on the state of the network. In this paper, we examine and report on our experience over the past several years with such mechanisms for videoconferencing software. We illustrate our points with results obtained with the IVS software developed at INRIA. We consider in particular rate and error control mechanisms. These mechanisms adapt the bandwidth requirements and the resilience to packet loss of the video stream sent by a source coder. We have found that they do prevent video sources from swamping the resources of the Internet, and that...

We consider the transmission of QCIF resolution (176 \Theta 144 pixels) video signals over wireless channels at transmission rates of 64 kbit/s and below. The bursty nature of the errors on the wireless channel requires careful control of transmission performance without unduly increasing the overhead for error protection. A dual-rate source coder is presented that adaptively selects a coding rate according to the current channel conditions. An ARQ (Automatic Repeat reQuest) error control technique is employed to retransmit erroneous data-frames. The source coding rate is selected based on the occupancy level of the ARQ transmission buffer. Error detection followed by retransmission results in less overhead than forward error correction for the same quality. Simulation results are provided for the statistics of the frame-error bursts of the proposed system over CDMA channels with average bit error rates of 10 \Gamma3 to 10 \Gamma4 . Parts of this paper have been presente...

Anecdotal evidence suggests that the quality of many videoconferences in the Internet is mediocre because of high packet loss rates. This makes it important to de sign and implement mechanisms that minimize packet loss and its impact in video (and audio) applications. There are two such types of mechanisms. Rate control mechanisms attempt to minimize the amount of packet loss by matching the bandwidth requirements of a video flow to the capacity available in the network. However, they do not prevent packet loss altogether. Error control mechanisms attempt to minimize the visual impact of lost packets at the destinations. In this paper, we provide motivation for using error control mechanisms based on forward error correction (FEC) and packet reconstruction. We examine a specific mechanism, and evaluate its cost as well as the benefit expected from using it. This mechanism can be augmented to obtain a joint source/channel coding scheme suitable for both the current and the future integra...

Real-time audio and video data streamed over unreliable IP networks, such as the Internet, may encounter losses due to dropped packets or late arrivals. This paper reviews error-concealment schemes developed for streaming realtime audio and video data over the Internet. Based on their interactions with (video or audio) source coders, we classify existing techniques into source coder-independent schemes that treat underlying source coders as black boxes, and source coder-dependent schemes that exploit coder-specific characteristics to perform reconstruction. Last, we identify possible future research directions. 1. Introduction Increases in bandwidth and computational speed lead to growing interests in real-time audio and video transmissions over the Internet. In the Internet, packets carrying real-time data may be dropped or arrive too late to be useful because the Internet is a packet-switched, best-effort delivery service, with no guarantee on the quality of service (QoS). Traditi...

In this paper we describe two error-recovery approaches for MPEG encoded video over ATM networks. The first approach aims at reconstructing each lost pixel by spatial interpolation from the nearest undamaged pixels. The second approach recovers lost macroblocks by minimizing intersample variations within each block and across its boundaries. Moreover, a new technique for packing ATM cells with compressed data is also proposed.

When transmitting compressed video over a data network, one has to deal with how channel errors affect the decoding process. This is particularly problematic with data loss or erasures. In this paper we describe techniques to address this problem in the context of networks where channel errors or congestion can result in the loss of entire macroblocks when MPEG video is transmitted. We describe spatial and temporal techniques for the recovery of lost macroblocks. In particular, we develop estimation techniques for the reconstruction of missing macroblocks using a Markov Random Field model. We show that the widely used heuristic motion compensated error concealment technique based on averaging motion vectors is a special case of our estimation technique. We further describe a technique that can be implemented in real-time.

Abstract—When transmitting compressed video over a data network, one has to deal with how channel errors affect the decoding process. This is particularly a problem with data loss or erasures. In this paper we describe techniques to address this problem in the context of Asynchronous Transfer Mode (ATM) networks. Our techniques can be extended to other types of data networks such as wireless networks. In ATM networks channel errors or congestion cause data to be dropped, which results in the loss of entire macroblocks when MPEG video is transmitted. In order to reconstruct the missing data, the location of these macroblocks must be known. We describe a technique for packing ATM cells with compressed data, whereby the location of missing macroblocks in the encoded video stream can be found. This technique also permits the proper decoding of correctly received macroblocks, and thus prevents the loss of ATM cells from affecting the decoding process. The packing strategy can also be used for wireless or other types of data networks. We also describe spatial and temporal techniques for the recovery of lost macroblocks. In particular, we develop several optimal estimation techniques for the reconstruction of missing macroblocks that contain both spatial and temporal information using a Markov random field model. We further describe a sub-optimal estimation technique that can be implemented in real time. Index Terms—ATM, cell loss, cell packing, error concealment, motion vectors, Markov random field, spatial reconstruction, temporal reconstruction. I.

In this paper we review error resilience techniques for real-time video transport over unreliable networks. Topics covered include an introduction to today's protocol and network environments and their characteristics, encoder error resilience tools, decoder error concealment techniques, as well as techniques that require cooperation between encoder, decoder and the network. We provide a review of general principles of these techniques as well as specific implementations adopted by the H.263 and MPEG-4 video coding standards. The majority of the paper is devoted to the techniques developed for block-based hybrid coders using motion-compensated prediction and transform coding. A separate section covers error resilience techniques for shape coding in MPEG-4. I. Introduction A. Error Resilience in Video Communications: Importance and Approach A video communications system typically involves five steps, as shown in Figure 1. The video is first compressed by a video encoder to reduce th...