Abstract — There are many potential applications for very highspeed vision sensors in robotics. Maybe tracking is the most obvious one. The main idea of this paper is to combine existing standard technology (CMOS imaging sensors, FPGA “glue logic”, and USB 2.0 interface) to use direct pixel access capabilities for real-time tracking. We designed and built such a prototype camera system, including FPGA programmed functionality for Fixed Pattern Noise (FPN) calibration, subsampling and direct subwindow access. The performance of this new camera is on one hand limited by the maximum pixel clock of the sensor, on the other hand by the USB 2.0 microframe timing and bandwith constraints. We achieve “frame rates ” of up to 2.5 kHz for small subwindows which can be randomly and individually addressed for each update cycle. Besides all given technical details and specifications of the system, we show a demonstration application of a high-speed blob tracking system which verifies the usability of our new camera for highly demanding tracking applications.

Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the author. Logarithmic CMOS image sensors capture high dynamic range scenes without saturation or loss of perceptible detail but problems exist with image quality. This thesis develops and applies methods of modelling and calibration to understand and improve the fixed pattern noise (FPN) and colour rendition of logarithmic imagers. Chapter 1 compares CCD and CMOS image sensors and, within the latter category, compares linear and logarithmic pixel designs. Chapter 2 reviews the literature on multilinear algebra, unifying and extending approaches for analytic and numeric manipulation of multi-index arrays, which are the generalisation of scalars, vectors and matrices. Chapter 3 defines and solves the problem of multilinear regression with linear constraints for the calibration of a sensor array, permitting models with linear relationships of parameters

At present, most CMOS image sensors use an array of pixels with a linear response. However, logarithmic CMOS sensors are also possible, which are capable of imaging high dynamic range scenes without saturating. Unfortunately, logarithmic sensors suffer from fixed pattern noise (FPN). Work reported in the literature generally assumes the FPN is independent of illumination. This paper develops a nonlinear model y = a + bln(c +x) of the pixel response y to an illuminance x showing that FPN arises from variation of the offset a, gain b and bias c. Equations are derived, which can be used to extract these parameters by calibration against a uniform illuminance of varying intensity. Experimental results, demonstrating parameter calibration and FPN correction, show that the nonlinear model outperforms previous models that assume either only offset or offset and gain variation. Keywords -- CMOS image sensors, logarithmic pixels, fixed pattern noise.