To enlarge an image, bi-linear interpolation and bi-cubic interpolation are two common and simple methods, but they always generate unacceptable zigzag or blurry results. Some image processing methods are provided to reduce the zigzag or blurry effects by enhancing the edges and textureness or deblurring the images, and some example-based methods used some training high resolution images to provide the missing high frequency part for the enlarged input low resolution image. Although it is possible to record the relationship between the high resolution and low resolution samples of some natural training images and transfer the relationship to enhance the enlarged input low resolution image by using the example-based methods, the noise embedded in the input image is also amplified and makes the enhanced enlarged image looks noisy. Due to the noise, the example-based methods cannot be used to produce a satisfying result when applying to video frames directly. In this paper, we present an improved example-based approach that records the relationship of the middle and high frequency data of some training images, since the low frequency data is not needed for reconstructing the high frequency one. When enlarging an image, we only use the middle frequency data to estimate the missing high frequency one and transfer it to enhance the textureness of the enlarged input image. Since our approach can reduce the noise while enhancing the enlarged image, it can also be used for enlarging the video frames directly. 1

In this work, we address the task of enhancing the spatial resolution of video sequences, known as super-resolution. Specifically, we consider the problem of superresolving a human face video by a very high (16) zoom factor. Inspired by recent literature on hallucination and example-based learning, we formulate this task using a graphical model that encodes 1) spatio-temporal consistencies, and 2) image formation & degradation processes. A video database of facial expressions is used to learn a domain-specific prior for high-resolution videos. The problem is now one of probabilistic inference, in which we aim to find the high resolution video that best satisfies the constraints expressed through the graphical model. Traditional approaches to this problem using video data first estimate the relative motion between frames and then compensate for it, resulting effectively in multiple measurements of the scene. Our use of time is rather direct: We define data structures that span multiple consecutive frames, enriching our feature vectors with a temporal signature. We then exploit these signatures to find consistent solutions over time. We present

Abstract — In this paper, the problem of recovering a highresolution frame from a sequence of low-resolution frames is considered. High-resolution reconstruction process highly depends on image registration step. Typical resolution enhancement techniques use global motion estimation technique. However, in general, video frames cannot be related through global motion due to the arbitrary individual pixel movement between frame pairs. To overcome this problem, we propose to employ segmentation-based optical flow estimation technique for motion estimation with a modified model for frame alignment. To do that, we incorporate the segmentation with the optical flow estimation in two-stage optical flow estimation. In the first stage, a reference image is segmented into homogeneous regions. In the second stage, the optical flow is estimated for each region rather than pixels or blocks. Then, the frame alignment is accomplished by optimizing the cost function that consists of L 1-norm of the difference between the interpolated low-resolution (LR) frames and the simulated LR frames. The experimental results demonstrate that using segmentation-based optical flow estimation in motion estimation step with the modified alignment model works better than other motion models such as affine, and conventional optical flow motion models. Keywords- Optical flow; image segmentation; Horn-Schunck; super resolution; resolution enhancement. I.