Robotic systems are becoming smaller, lower power, and cheaper, enabling their application in areas not previously considered. This is true of vision systems as well. SRI's Small Vision Module (SVM) is a compact, inexpensive realtime device for computing dense stereo range images, which are a fundamental measurement supporting a wide range of computer vision applications. We describe hardware and software issues in the construction of the SVM, and survey implemented systems that use a similar area correlation algorithm on a variety of hardware. 1 Introduction Realtime stereo analysis, until recently, has been implemented in large custom hardware arrays (Kanade 1996, Matthies 1995). But computational power and algorithmic advances have made it possible to do such analysis on single processors. At the same time, increased density, speed and programmability of floating-point gate arrays (FPGAs) make custom hardware a viable alternative. In this paper, we discuss the implementation of ar...

A stereo vision system that is required to support high-level object based tasks in a tele-operated environment is described. Stereo vision is computationally expensive, due to having to find corresponding pixels. Correlation is a fast, standard way to solve the correspondence problem. This paper analyses the behaviour of correlation based stereo to find ways to improve its quality while maintaining its realtime suitability. Three methods are suggested. Two of them aim to improve the disparity image especially at depth discontinuities, while one targets the identification of possible errors in general. Results are given on real stereo images with ground truth. A comparison with five standard correlation methods shows that improvements of simple stereo correlation are possible in real-time on current computer hardware.

orting a C compiler to the DISC processor. Justin Diether assisted in the design, hand-layout, and testing of many partially reconfigured circuits. I would also like to thank Paul Graham for his generous assistance and support of our many mutual activities, classes, and projects at BYU. Other graduate students assisting me with this work include Russel Peterson, Mike Rencher, Richard Ross, and Peter Bellows. My advisor, Brad Hutchings, provided essential assistance and encouragement in all of the projects, ideas, and results presented within this work. My decision to complete this degree and write this dissertation was influenced largely by his advice and positive encouragement. Brent Nelson and other faculty members within the Electrical and Computer Engineering department at BYU have provided critical feedback on a wide variety of topics relating to this work. I would also like to acknowledge the insight and assistance of many collaborators researching closely related subjects. For

We develop a fast and accurate variable window approach. The two main ideas for achieving accuracy are choosing a useful range of window sizes/shapes for evaluation and developing a new window cost which is particularly suitable for comparing windows of different sizes. The speed of our approach is due to the Integral Image technique, which allows computation of our window cost over any rectangular window in constant time, regardless of window size. Our method ranks in the top four on the Middlebury stereo database with ground truth, and performs best out of methods which have comparable efficiency.

Current processors are programmed through a fixed interface called the Instruction Set Architecture (ISA). Consequently, a compiler targeting such a processor is forced to choose instructions from the provided instruction set while generating code for a given application. Often this instruction set is not a suitable match for the computational requirements of the application program. With in this context, we ask ourselves the following questions. 1. Can application performance be improved if the compiler had the freedom to pick the instruction set on a per application basis? 2. Can we build cost-effective processors that provide the ability to efficiently emulate compiler determined instruction sets and yet are not application specific? 3. Given that the desired processor capabilities are feasible, can the compiler determine an optimal set of instructions for a given application and generate code that can effectively exploit the processor capabilities? In this thesis, we provide sufficient evidence to answer these questions in the affirmative. Through a combination of architectural innovations and novel compilation techniques, this dissertation demonstrates that it is possible to attain significant improvement in performance, up to an order of magnitude in some cases, on general purpose and multimedia applications over comparable fixed ISA processors. We propose classes of microprocessors that allow application programs to add and subtract functional units yielding a dynamically varying instruction set interface to the running application without compromising current compatibility model. First half of this dissertation describes this novel class of architectures, focusing on a specific subclass called Adaptive Explicitly Parallel Instruction Computing (AEPIC) architectures...

This paper presents a fast panoramic stereo matching algorithm using a cylindrical maximum surface technique. The disparity for a pair of panoramic images is found in a cylindrical shaped correlation coefficient volume by obtaining the maximum surface rather than simply choosing a position that gives the maximum correlation coefficient value. The use of our cylindrical maximum surface technique ensures that the disparities obtained at the left and the right columns of the panoramic stereo images are properly constrained. Typical running time for a pair of 1324120 images is about one third of a second on a 1.7GHz PC. A variety of real images have been tested, and good results have been obtained.

The processing power and network bandwidth required for true immersive telepresence applications are only now beginning to be available. We draw from our experience developing stereo based tele-immersion prototypes to present the main issues arising when building these systems. Tele-immersion is a new medium that enables a user to share a virtual space with remote participants. The user is immersed in a rendered 3D-world that is transmitted from a remote site. To acquire this 3D model we apply bi- and trinocular stereo techniques which provide a view-independent scene description. Slow processing cycles or long network latencies interfere with the users' ability to communicate, so the dense stereo range data must be computed and transmitted at high frame rates. Moreover, reconstructed 3D views of the remote scene must be as accurate as possible to achieve a sense of presence. We address both issues of speed and accuracy using a variety of techniques including the power of supercomputing clusters and a method for combining motion and stereo in order to increase speed and robustness. We present the latest prototype acquiring a room-size environment in realtime using a supercomputing cluster and we discuss its strengths and current weaknesses.

Stereo matching is important in the area of computer vision and photogrammetry. This paper presents a fast and reliable stereo matching algorithm which produces a dense disparity map by using a pyramid structure, fast cross-correlation, rectangular subregioning and dynamic programming techniques. Fast correlation is achieved by using the box filtering technique which is invariant to the size of the correlation window, and by segmenting the images at different levels of the pyramid into rectangular subimages. By working with rectangular subimages, the speed of the algorithm can be increased and the intermediate memory storage required is reduced. The disparity for each scan line is found in the correlation matrix by finding the best path using dynamic programming rather than simply choosing the position that gives the maximum correlation coefficient. Typical running time for a 512\Theta512 image is in the order of half a minute rather than minutes or hours. A variety of synthetic and re...

Optical flow or image motion estimation is important in the area of computer vision. This paper presents a fast and reliable optical flow algorithm which produces a dense optical flow map by using fast cross-correlation and shortest-path techniques. Fast correlation is achieved by using the box filtering technique which is invariant to the size of the correlation window. The motion for each scan line of the input image is obtained from the correlation volume by finding the best 3D path using dynamic programming rather than simply choosing the position that gives the maximum cross correlation coefficient. Sub-pixel accuracy is achieved by fitting the local correlation coefficients to a quadratic surface. Typical running time for a 256\Theta256 image is in the order of a few seconds rather than minutes. A variety of synthetic and real images have been tested, and good results have been obtained. 1. Introduction Optical flow or image motion is the displacement of each image pixels in an...

This paper presents a fast and reliable stereo matching algorithm which produces a dense disparity map by using fast cross-correlation, rectangular subregioning and maximumsurface techniques in a coarse-to-fine scheme. Fast correlation is achieved by using the box filtering technique whose speed is invariant to the size of the correlation window, and by segmenting the images at different levels of the pyramid into rectangular subimages. By working with rectangular subimages, the speed of the algorithm can be increased and the intermediate memory storage requirement is reduced. The disparity for the whole image is found in the correlation coefficient volume by obtaining the maximum-surface rather than simply choosing the position that gives the maximum correlation coefficient value. Typical running time for a 512\Theta512 image is in the order of half a minute rather than minutes or hours. A variety of synthetic and real images have been tested, and good results have been obtained. 1.