The problem of error control and concealment in video communication is becoming increasingly important because of the growing interest in video delivery over unreliable channels such as wireless networks and the Internet. This paper reviews the techniques that have been developed for error control and concealment in the past ten to fifteen years. These techniques are described in three categories according to the roles that the encoder and decoder play in the underlying approaches. Forward error concealment includes methods that add redundancy at the source end to enhance error resilience of the coded bit streams. Error concealment by postprocessing refers to operations at the decoder to recover the damaged areas based on characteristics of image and video signals. Finally, interactive error concealment covers techniques that are dependent on a dialog between the source and destination. Both current research activities and practice in international standards are covered.

We study the robust estimation of missing regions in images and video using adaptive, sparse reconstructions. Our primary application is on missing regions of pixels containing textures, edges, and other image features that are not readily handled by prevalent estimation and recovery algorithms. We assume that we are given a linear transform that is expected to provide sparse decompositions over missing regions such that a portion of the transform coe#cients over missing regions are zero or close to zero. We adaptively determine these small magnitude coe#cients through thresholding, establish sparsity constraints, and estimate missing regions in images using information surrounding these regions. Unlike prevalent algorithms, our approach does not necessitate any complex preconditioning, segmentation, or edge detection steps, and it can be written as a sequence of denoising operations. We show that the region types we can e#ectively estimate in a mean squared error sense are those for which the given transform provides a close approximation using sparse nonlinear approximants. We show the nature of the constructed estimators and how these estimators relate to the utilized transform and its sparsity over regions of interest. The developed estimation framework is general, and can readily be applied to nonstationary signals with a suitable choice of linear transforms. Part I discusses fundamental issues, and Part II is devoted to adaptive algorithms with extensive simulation examples that demonstrate the power of the proposed techniques.

This paper first proposes a computationally efficient spatial directional interpolation scheme which makes use of the local geometric information extracted from the surrounding blocks. The proposed error concealment scheme produces results that are superior to those of other approaches, in terms of both peak signal-to-noise ratio and visual quality. Then a novel approach which incorporates this directional spatial interpolation at the receiver is proposed for block-based low bit rate coding. The key observation is that the directional spatial interpolation at the receiver can reconstruct faithfully a large percentage of the blocks that are intentionally not sent. Rate-distortion optimal way to drop the blocks is shown. The new approach can be made compatible with standard JPEG and MPEG decoders. The block-dropping approach also has an important application for dynamic rate shaping in transmitting precompressed videos over channels of dynamic bandwidth. Experimental results show that the proposed coding and rate shaping systems can provide significant subjective and objective gains over conventional approaches. 1

This paper introduces a new framework for error concealment in block-based image coding systems: sequential recovery. Unlike previous approaches that simultaneously recover the pixels inside a missing block, we propose to recover them in a sequential fashion such that the previously-recovered pixels can be used in the recovery process afterwards. The principal advantage of the sequential approach is the improved capability of recovering important image features brought by the reduction in the complexity of statistical modeling, i.e., from blockwise to pixelwise. Under the framework of sequential recovery, we present an orientation adaptive interpolation scheme derived from the pixelwise statistical model. We also investigate the problem of error propagation with sequential recovery and propose a linear merge strategy to alleviate it. Extensive experiment results are used to demonstrate the improvement of the proposed sequential error-concealment technique over previous techniques in the literature.

A new class of image information-restoration algorithms virtually different from traditional techniques are proposed. In comparison with other approaches, our methods not only use the information in local areas, but also that in the remote regions in the image. The methods originate from the idea that there exists abundant long-range correlation within natural images and the human vision systems composed of our eyes and brains can sufficiently utilize such types of information redundancy to implement the functions of image interpretation, representation, restoration, enhancement, and error concealment. Our general approach can be summarized as five basic steps: fetching, searching, matching, competing, and recovering. The experimental results on several practical applications show that our methods perform substantially better than many other state-of-the-art methods.

The MPEG-2 compression algorithm is very sensitive to channel disturbances due to the use of variable length coding. A single bit error during transmission leads to noticeable degradation of the decoded sequence quality in that part or entire slice information is lost until the next resynchronization point is reached. Error concealment (EC) methods, implemented at the decoder side, present one way of dealing with this problem. An error concealment scheme that is based on block matching principles and spatio-temporal video redundancy is presented in this paper. Spatial information (for the first frame of the sequence or the next scene) or temporal information (for the other frames) is used to reconstruct the corrupted regions. The concealment strategy is embedded in the MPEG-2 decoder model, in such a way that error concealment is applied after entire frame decoding. Its performance proves to be satisfactory for packet error rates (PER) ranging from 1% to 10% and for video sequ...

The problem of errors occurring in MPEG-2 coded video sequences, caused by signal loss during transmission, is examined in this paper and an attempt is made to reconstruct the lost parts at each frame. The proposed error concealment scheme exploits reconstructed temporal information from previously decoded frames in order to conceal bitstream errors in all types of frames: I, P, or B, as long as temporal information is available. Since no such information is available for the first frame (I-frame) of an MPEG-2 coded sequence, another concealment technique is added to the proposed scheme, which uses spatial information from neighbouring macroblocks (MBs). The simulation results compared with other methods prove to be better judging from both PSNR values and the perceived visual quality of the reconstructed sequence. Its quality ameliorates with time. I. Introduction The MPEG-2 compression algorithm allows compressed bitstreams at bitrates close to 20 Mbits/s. These are transmitted th...

Abstract—The transmission of block-coded visual information over packet networks introduces fidelity problems in terms of data losses, which result in wrong reconstruction of block sequences at the decoder. Concealment techniques aim at masking the visual effect of these errors, by exploiting either spatial or temporal available information. Both temporal and spatial approaches present drawbacks: the first is in general inefficient in handling complex or fast objects ’ motion, while the second is computationally expensive and is not able to recover high-frequency contents and small details. In this paper, a new solution is proposed that combines temporal and spatial approaches. The technique first replaces the lost block with the best matching pattern in a previously decoded frame (BMA), using the border information, and then applies a meshbased warping (MBW) that reduces the artifacts caused by fast movements, rotations or deformations. The first step is achieved by a fast matching algorithm, for a high precision is not needed, while the second step uses an affine transform applied to a deformable mesh structure. Experimental results show that significant improvements can be achieved in comparison with traditional spatial or temporal concealment approaches, in terms of both subjective and objective reconstruction quality. Index Terms—Affine transform, boundary matching algorithm, mesh-based processing, spatial and temporal concealment, video concealment. I.

Abstract—Compressed video sequences are very vulnerable to channel disturbances when they are transmitted through an unreliable medium such as a wireless channel. Transmission errors not only corrupt the current decoded frame, but they may also propagate to succeeding frames. A number of post-processing error concealment (ECN) methods that exploit the spatial and/or temporal redundancy in the video signal have been proposed to combat channel disturbances. Although these approaches can effectively conceal lost or erroneous macroblocks (MBs), all of them only consider spatial and/or temporal correlation in a single frame (the corrupted one), which limits their ability to obtain an optimal recovery. Since the error propagates to the next few motion-compensated frames in the presence of lost MBs in an or frame, error concealment should simultaneously minimize the errors not only in the current decoded frame but also in the succeeding and frames that depend on the corrupted frame. In this paper, we propose a novel multiframe recovery principle which analyzes the propagation of a lost MB into succeeding frames. Then, MPEG-compatible spatial and temporal error concealment approaches using this multiframe recovery principle are proposed, where the lost MBs are recovered in such a way that the error propagation is minimized. Index Terms—Error concealment, error propagation, error-resilient video communications, MPEG, multiframe processing. I.

this paper we describe a system to show some limited effects on a static toy-car model and present techniques that can be used in similar setups. Our focus is on creating apparent motion for animation