Oriented filters are useful in many early vision and image processing tasks. One often needs to apply the same filter, rotated to different angles under adaptive control, or wishes to calculate the filter response at various orientations. We present an efficient architecture to synthesize filters of arbitrary orientations from linear combinations of basis filters, allowing one to adaptively "steer" a filter to any orientation, and to determine analytically the filter output as a function of orientation.

Orthogonal wavelet transforms have recently become a popular representation for multiscale signal and image analysis. One of the major drawbacks of these representations is their lack of translation invariance: the content of wavelet subbands is unstable under translations of the input signal. Wavelet transforms are also unstable with respect to dilations of the input signal, and in two dimensions, rotations of the input signal. We formalize these problems by defining a type of translation invariance that we call "shiftability". In the spatial domain, shiftability corresponds to a lack of aliasing; thus, the conditions under which the property holds are specified by the sampling theorem. Shiftability may also be considered in the context of other domains, particularly orientation and scale. We explore "jointly shiftable" transforms that are simultaneously shiftable in more than one domain. Two examples of jointly shiftable transforms are designed and implemented: a one-dimensional tran...

Abstract. This note is a very basic introduction to wavelets. It starts with an orthogonal basis of piecewise constant functions, constructed by dilation and translation. The “wavelet transform ” maps each f(x) to its coefficients with respect to this basis. The mathematics is simple and the transform is fast (faster than the Fast Fourier Transform, which we briefly explain), but approximation by piecewise constants is poor. To improve this first wavelet, we are led to dilation equations and their unusual solutions. Higher-order wavelets are constructed, and it is surprisingly quick to compute with them — always indirectly and recursively. We comment informally on the contest between these transforms in signal processing, especially for video and image compression (including high-definition television). So far the Fourier Transform — or its 8 by 8 windowed version, the Discrete Cosine Transform — is often chosen. But wavelets are already competitive, and they are ahead for fingerprints. We present a sample of this developing theory. 1. The Haar wavelet To explain wavelets we start with an example. It has every property we hope for, except one. If that one defect is accepted, the construction is simple and the computations are fast. By trying to remove the defect, we are led to dilation equations and recursively defined functions and a small world of fascinating new problems — many still unsolved. A sensible person would stop after the first wavelet, but fortunately mathematics goes on. The basic example is easier to draw than to describe: Figure 1. Scaling function φ(x), wavelet W(x), and the next level of detail.

We present MikeTalk, a text-to-audiovisual speech synthesizer which converts input text into an audiovisual speech stream. MikeTalk is built using visemes, which are a small set of images spanning a large range of mouth shapes. The visemes are acquired from a recorded visual corpus of a human subject which is specifically designed to elicit one instantiation of each viseme. Using optical flow methods, correspondence from every viseme to every other viseme is computed automatically. By morphing along this correspondence, a smooth transition between viseme images may be generated. A complete visual utterance is constructed by concatenating viseme transitions. Finally, phoneme and timing information extracted from a text-to-speech synthesizer is exploited to determine which viseme transitions to use, and the rate at which the morphing process should occur. In this manner, we are able to synchronize the visual speech stream with the audio speech stream, and hence give the impression of a photorealistic talking face.

In this thesis, I investigate issues in the development of continuous media (CM) applications. CM applications process, transport, or store CM data such as digital audio and video. Introduction of video into applications will require support for sophisticated video effects such as image processing, composition and digital filtering. Traditional image processing and composition algorithms operate on uncompressed images, while video is typically stored and delivered in a compressed form. High computational cost, along with the complexity of video decompression, makes traditional algorithms too slow for interactive use on video data. This thesis describes and evaluates a new family of algorithms for computing video effects that run one to two orders of magnitude faster than their ...

We describe a technique for displaying patterns that appear to move continuously without changing their positions. The method uses a quadrature pair of oriented filters to vary the local phase, giving the sensation of motion. We have used this technique in various computer graphic and scientific visualization applications.

Computational mechanics, an approach to structural complexity, defines a process’s causal states and gives a procedure for finding them. We show that the causal-state representation—an E-machine—is the minimal one consistent with

Two paradigms for visual analysis are top-down, starting from high-level models or information about the image, and bottom-up, where little is assumed about the image or objects in it. We explore a local, bottom-up approach to image analysis. We develop operators to identify and classify image junctions, whichcontain important visual cues for identifying occlusion, transparency, and surface bends. Like the human visual system, we begin with the application of linear filters which are oriented in all possible directions. Wedevelop an efficientway to create an oriented filter of arbitrary orientation by describing it as a linear combination of basis filters. This approach to oriented filtering, which we call steerable filters, offers advantages for analysis as well as computation. We design a variety of steerable filters, including steerable quadrature pairs, which measure local energy. We show applications of these filters in orientation and texture analysis, and image representation and enhanc...

This thesis addresses a number of issues concerned with the restoration of one type of image sequence, namely archived black and white motion pictures. These are often a valuable historical record, but because of the physical nature of the film they can suffer from a variety of degradations which reduce their usefulness. The main visual defects are `dirt and sparkle' due to dust and dirt becoming attached to the film, or abrasion removing the emulsion, and `line scratches' due to the film running against foreign bodies in the camera or projector. For an image

This paper describes a new technique to implement operations on compressed digital video images that allows many image manipulation operations to be performed 50 to 100 times faster than the corresponding algorithms operating on uncompressed images. This is accomplished by performing the operations directly on the compressed data in the DCT domain. In this paper, we show how to transform image space operations into DCT space operations, we describe several representative algorithms in the DCT domain and report their performance. 1. Introduction Multimedia applications that operate on audio and video data will enable many new applications of computers. For example, a collaborative work system can include video conferencing windows, or a hypermedia training system can include audio and video instructional material. While most research on multimedia applications has focused on compression standards ([3], [9], [10], [14]), synchronization issues ([11], [12]), storage representations ([12...