Abstract—Advances in CMOS image sensors enable high-speed image readout, which makes it possible to capture multiple images within a normal exposure time. Earlier work has demonstrated the use of this capability to enhance sensor dynamic range. This paper presents an algorithm for synthesizing a high dynamic range, motion blur free, still image from multiple captures. The algorithm consists of two main procedures, photocurrent estimation and saturation and motion detection. Estimation is used to reduce read noise, and, thus, to enhance dynamic range at the low illumination end. Saturation detection is used to enhance dynamic range at the high illumination end as previously proposed, while motion blur detection ensures that the estimation is not corrupted by motion. Motion blur detection also makes it possible to extend exposure time and to capture more images, which can be used to further enhance dynamic range at the low illumination end. Our algorithm operates completely locally; each pixel’s final value is computed using only its captured values, and recursively, requiring the storage of only a constant number of values per pixel independent of the number of images captured. Simulation and experimental results demonstrate the enhanced signal-to-noise ratio (SNR), dynamic range, and the motion blur prevention achieved using the algorithm. Index Terms—CMOS image sensor, dynamic range extension, motion blur restoration, motion detection, photocurrent estimation, saturation detection. I.

Most of today's video and digital cameras use CCD image sensors, where the electric charge collected by the photodetector array during exposure time is serially shifted out of the sensor chip resulting in slow readout speed and high power consumption. Recently developed CMOS image sensors, by comparison, are read out non-destructively and in a manner similar to a digital memory and can thus be operated at very high frame rates. A CMOS image sensor can also be integrated with other camera functions on the same chip ultimately leading to a single-chip digital camera with very compact size, low power consumption and additional functionality. CMOS image sensors, however, generally su#er from lower dynamic range than CCDs due to their high read noise and non-uniformity. Moreover, as sensor design follows CMOS technology scaling, well capacity will continue to decrease, eventually resulting in unacceptably low SNR.

Advances in CMOS image sensors enable fast image capture, which makes it possible to capture multiple images within a normal exposure time. An algorithm that takes advantage of this capability by simultaneously constructing a high dynamic range image and performing motion blur restoration from multiple image captures is described. The algorithm comprises two main procedures -- photocurrent estimation and motion /saturation detection. It operates completely locally -- each pixel's final value is computed using only its captured values, and recursively, requiring the storage of only a constant number of values per pixel independent of the number of images captured. These modest computational and storage requirements make it feasible to integrate all needed memory and processing with the image sensor on a single CMOS chip. Simulation results demonstrate the enhanced SNR, dynamic range, and the motion blur restoration obtained using our algorithm. 1.

The market for solid-state image sensors has been experiencing explosive growth in recent years due to the increasing demands of mobile imaging, digital still and video cameras, Internet-based video conferencing, surveillance, and biometrics. With over 230 million parts shipped in 2004 and an estimated annual growth rate of over 28 % (In-Stat/MDR), image sensors have become a significant silicon technology driver. Charge-coupled devices (CCDs) have traditionally been the dominant image-sensor technology. Recent advances in the design of image sensors implemented in complementary metaloxide semiconductor (CMOS) technologies have led to their adoption in several high-volume products, such as the optical mouse, PC cameras, mobile phones, and high-end digital cameras, making them a viable alternative to CCDs. Additionally, by exploiting the ability to integrate sensing with analog and digital processing down to the pixel level, new types of CMOS imaging devices are being created for manmachine interface, surveillance and monitoring, machine vision, and biological testing, among other applications. In this article, we provide a basic introduction to CMOS image-sensor technology, design, and performance limits and present recent