## Single-pixel imaging via compressive sampling

Venue: | IEEE Signal Processing Magazine |

Citations: | 155 - 12 self |

### BibTeX

@ARTICLE{Duarte_single-pixelimaging,

author = {Marco F. Duarte and Mark A. Davenport and Dharmpal Takhar and Jason N. Laska and Ting Sun and Kevin F. Kelly and Richard G. Baraniuk},

title = {Single-pixel imaging via compressive sampling},

journal = {IEEE Signal Processing Magazine},

year = {},

volume = {2008}

}

### OpenURL

### Abstract

Humans are visual animals, and imaging sensors that extend our reach – cameras – have improved dramatically in recent times thanks to the introduction of CCD and CMOS digital technology. Consumer digital cameras in the mega-pixel range are now ubiquitous thanks to the happy coincidence that the semiconductor material of choice for large-scale electronics integration (silicon) also happens to readily convert photons at visual wavelengths into electrons. On the contrary, imaging at wavelengths where silicon is blind is considerably more complicated, bulky, and expensive. Thus, for comparable resolution, a $500 digital camera for the visible becomes a $50,000 camera for the infrared. In this paper, we present a new approach to building simpler, smaller, and cheaper digital cameras that can operate efficiently across a much broader spectral range than conventional silicon-based cameras. Our approach fuses a new camera architecture based on a digital mi-cromirror device (DMD – see Sidebar: Spatial Light Modulators) with the new mathematical theory and algorithms of compressive sampling (CS – see Sidebar: Compressive Sampling in a Nutshell). CS combines sampling and compression into a single nonadaptive linear measurement process [1–4]. Rather than measuring pixel samples of the scene under view, we measure inner products

### Citations

1869 | Compressed sensing - Donoho - 2006 |

888 | Near-optimal signal recovery from random projections: Universal encoding strategies
- Candès, Tao
(Show Context)
Citation Context ...olor filters and a photomultiplier tube with M = N/10. In both cases, the images were reconstructed using Total Variation minimization, which is closely related to wavelet coefficient ℓ1 minimization =-=[2]-=-. Structured illumination configuration In a reciprocal configuration to that in Fig. 1, we can illuminate the scene using a projector displaying a sequence of random patterns {φm} and collect the ref... |

804 |
Stable signal recovery from incomplete and inaccurate measurements
- Candes, Romberg, et al.
- 2006
(Show Context)
Citation Context ...Nutshell), we require up to an additional log 2 CN bits per measurement. One empirical study has found roughly that CN lies between 8 and 10 for a range of different random measurement configurations =-=[15]-=-. Thus, BS and CS require a higher-resolution A/D converter than PA and RS to acquire an image with the same worst-case quantization distortion. 7sTABLE I Comparison of the four camera scanning method... |

692 |
Compressive sampling
- Candès
- 2006
(Show Context)
Citation Context ...e emphasize that in the CS case, we have only a fairly loose upper bound and that there exist alternative CS reconstruction algorithms with better performance guarantees, such as the Dantzig Selector =-=[3]-=-. Summary: From Table I, we see that the advantages of a single-pixel camera over a PA come at the cost of more stringent demands on the sensor dynamic range and A/D quantization and larger MSE due to... |

692 | Vector Quantization
- Gray
- 1984
(Show Context)
Citation Context ...s MSE = 1 N �x − �x�2 2 . Assuming that each measurement in the PA and RS is quantized to B bits, the worst-case mean-squared quantization error for the quantized PA and RS images is MSE = √ ND2 −B−1 =-=[14]-=-. Due to its larger dynamic range, BS requires B + log 2 N bits per measurement to reach the same MSE distortion level. Since the distortion of CS reconstruction is up to CN times larger than the dist... |

325 | Compressive sensing
- Baraniuk
- 2007
(Show Context)
Citation Context ...rstanding that a small collection of nonadaptive linear measurements of a compressible signal or image contain enough information for reconstruction and processing [1–3]. For a tutorial treatment see =-=[4]-=- or the paper by Romberg in this issue. The traditional approach to digital data acquisition samples an analog signal uniformly at or above the Nyquist rate. In a digital camera, the samples are obtai... |

320 | A simple proof of the restricted isometry property for random matrices
- Baraniuk, Davenport, et al.
- 2006
(Show Context)
Citation Context ...old search is an estimate of the template parameters that best match the test image. Previous work in the computer science community (the other “CS”) has also employed the Johnson-Lindenstrauss lemma =-=[22]-=- to reduce the data dimensionality prior to computing features for classification; however, they have not considered the intrinsic manifold structure manifested in many image processing and computer v... |

144 | Sparsity and incoherence in compressive sampling
- Candès, Romberg
- 2007
(Show Context)
Citation Context ...ession algorithm like JPEG or JPEG2000. Since the DMD array is programmable, we can also employ test functions φm drawn randomly from a fast transform such as a Walsh, Hadamard, or Noiselet transform =-=[6, 7]-=-. The single-pixel design reduces the required size, complexity, and cost of the photon detector array down to a single unit, which enables the use of exotic detectors that would be impossible in a co... |

86 |
Motion-compensated highly scalable video compression using an adaptive 3D wavelet transform based on lifting
- Secker, Taubman
- 2001
(Show Context)
Citation Context ...ces are often related temporally through smooth object motions in the video. Exploiting this 3D compressibility in both the space and time directions and inspired by modern 3D video coding techniques =-=[11]-=-, we can, for example, attempt to reconstruct the sparsest video space/time cube in the 3D wavelet domain. These two approaches are compared in the simulation study illustrated in Fig. 3. We employed ... |

80 | Random projections of smooth manifolds
- Baraniuk, Wakin
- 2007
(Show Context)
Citation Context ... a recent result that the structure of a smooth K-dimensional manifold in R N is preserved with high probability under a random projection to the lower dimensional space R M as long as M = O(K log N) =-=[21]-=-. This is reminiscent of the number of measurements required for successful CS but with K now the manifold dimension. Thus, to classify an N-pixel test image, we can alternatively compare M random mea... |

74 | A new compressive imaging camera architecture using optical-domain compression
- Takhar, Laska, et al.
- 2006
(Show Context)
Citation Context ...hen recovered or To appear in IEEE Signal Processing Magazine, Special Issue on Compressive Sampling. March 2008. 1sFig. 1. Aerial view of the single-pixel compressive sampling (CS) camera in the lab =-=[5]-=-. processed from the measurements by a digital computer. The camera design reduces the required size, complexity, and cost of the photon detector array down to a single unit, which enables the use of ... |

54 | Dual photography
- SEN, CHEN, et al.
- 2005
(Show Context)
Citation Context ...ntages in applications where we can control the light source. In particular, there are intriguing possible combinations of single-pixel imaging with techniques such as 3D imaging and dual photography =-=[9]-=-. 4sShutterless video imaging We can also acquire video sequences using the single-pixel camera. Recall that a traditional video camera opens a shutter periodically to capture a sequence of images (ca... |

50 | Compressive imaging for video representation and coding
- Wakin, Laska, et al.
- 2006
(Show Context)
Citation Context ...needs no shutter; we merely continuously sequence through randomized test functions φm and then reconstruct a video sequence using an optimization that exploits the video’s spatio-temporal redundancy =-=[10]-=-. If we view a video sequence as a 3D space/time cube, then the test functions φm lie concentrated along a periodic sequence of 2D image slices through the cube. A naïve way to reconstruct the video s... |

42 | The multiscale structure of non-differentiable image manifolds
- Wakin, Donoho, et al.
- 2005
(Show Context)
Citation Context ...ms all of its operations directly on the random measurements [18]. The two key elements of the smashed filter are the generalized likelihood ratio test and the concept of an image appearance manifold =-=[20]-=-. If the parametric transformation affecting the template image is well-behaved, then the set of transformed templates forms a low-dimensional manifold in the high-dimensional pixel space R N with the... |

41 | Sparse signal detection from incoherent projections - Duarte, Davenport, et al. - 2006 |

29 | Random lens imaging
- Fergus, Torralba, et al.
- 2006
(Show Context)
Citation Context ...k in [29] has compared several single and multiple pixel imaging strategies of varying complexities; their simulation 11sresults for Poisson counting noise agree closely with those above. Finally, in =-=[30]-=- the authors use CS principles and a randomizing lens to boost both the resolution and robustness of a conventional digital camera. V. Conclusions For certain applications, CS promises to substantiall... |

27 | Compressive sampling for signal classification - Haupt, Castro, et al. - 2006 |

14 |
Application of spatial light modulators for new modalities in spectrometry and imaging
- DeVerse, Coifman, et al.
(Show Context)
Citation Context ... a single sensor takes N light measurements sequentially from different combinations of the N pixels as determined by test functions {φm} that are not delta functions but from some more general basis =-=[12]-=-. In our analysis, we assume a Walsh basis modified to take the values 0/1 rather than ±1; thus Φ = W , where W is the 0/1 Walsh matrix. The acquired image is obtained from the measurements y by �x = ... |

12 | Multiplex sensors and the constant radiance theorem
- Brady
- 2002
(Show Context)
Citation Context ...nd M = 6500 random measurements. that the design inherits from the CS theory include its universality, robustness, and progressivity. The single-pixel design falls into the class of multiplex cameras =-=[8]-=-. The baseline standard for multiplexing is classical raster scanning, where the test functions {φm} are a sequence of delta functions δ[n − m] that turn on each mirror in turn. As we will see below, ... |

12 |
Sensor-layer image compression based on the quantized cosine transform
- Pitsianis, Brady, et al.
- 2005
(Show Context)
Citation Context ...atterns, which enables simple demultiplexing, and randomized Hadamard patterns, which yield a uniform signal-to-noise ratio among the measurements. Other compressive cameras developed to date include =-=[27, 28]-=-, which employ optical elements to perform transform coding of multispectral images. These designs obtain sampled outputs that correspond to coded information of interest, such as the wavelength of a ... |

11 | Virtual Sensor Design
- Costantini, Süsstrunk
- 2004
(Show Context)
Citation Context ...T (3N − 2) P T < 3C 2 NM P T Photon counting noise: In addition to quantization error from the A/D converter, each camera will also be affected by image-dependent Poisson noise due to photon counting =-=[16]-=-. We compare the MSE due to photon counting for each of the scanning methodologies. The details are worked out in Appendix: Poisson Photon Counting Calculations and are summarized in Table I. We see t... |

7 |
Confocal microendoscopy with chromatic sectioning
- Lane, Elliott, et al.
- 2003
(Show Context)
Citation Context ...0t T = t IV. Other Multiplexing Camera Architectures Two notable existing DMD-driven imaging applications involve confocal microscopy (which relates closely to the raster scan strategy studied above) =-=[23, 24]-=- and micro-optoelectromechanical (MOEM) systems [12, 25, 26]. In a MOEM system, a DMD is positioned between the scene and the sensing array to perform column-wise multiplexing by allowing only the lig... |

7 |
An Overview of the Digital Micromirror Device (DMD) and Its Application to Projection Displays
- Sampsell
- 1993
(Show Context)
Citation Context ...arency. Another example is a liquid crystal display (LCD) projector. The Texas Instruments (TI) digital micromirror device (DMD) is a reflective SLM that selectively redirects parts of the light beam =-=[31]-=-. The DMD consists of an array of bacteriumsized, electrostatically actuated micro-mirrors, where each mirror in the array is suspended above an individual static random access memory (SRAM) cell (see... |

5 | Hyper-spectral analysis of normal and malignant microarray tissue sections using a novel micro-optoelectricalmechanical system
- Davis, Maggioni, et al.
(Show Context)
Citation Context ... notable existing DMD-driven imaging applications involve confocal microscopy (which relates closely to the raster scan strategy studied above) [23, 24] and micro-optoelectromechanical (MOEM) systems =-=[12, 25, 26]-=-. In a MOEM system, a DMD is positioned between the scene and the sensing array to perform column-wise multiplexing by allowing only the light from the desired pixels to pass through. In [12, 25] the ... |

4 |
High-speed confocal laser scanning microscopy using acousto-optic deflectors and a digital micromirror device
- Bansal, Patel, et al.
- 2004
(Show Context)
Citation Context ...0t T = t IV. Other Multiplexing Camera Architectures Two notable existing DMD-driven imaging applications involve confocal microscopy (which relates closely to the raster scan strategy studied above) =-=[23, 24]-=- and micro-optoelectromechanical (MOEM) systems [12, 25, 26]. In a MOEM system, a DMD is positioned between the scene and the sensing array to perform column-wise multiplexing by allowing only the lig... |

4 |
Target detection using integrated hyper spectral sensing and processing
- Muise, Mahalanobis
- 2005
(Show Context)
Citation Context ... notable existing DMD-driven imaging applications involve confocal microscopy (which relates closely to the raster scan strategy studied above) [23, 24] and micro-optoelectromechanical (MOEM) systems =-=[12, 25, 26]-=-. In a MOEM system, a DMD is positioned between the scene and the sensing array to perform column-wise multiplexing by allowing only the light from the desired pixels to pass through. In [12, 25] the ... |

3 |
Compressive sampling versus conventional imaging
- Haupt, Nowak
- 2006
(Show Context)
Citation Context ...age performance in practice can be considerably better. Our results are summarized in Table I. An alternative analysis of CS imaging for piecewise smooth images in Gaussian noise has been reported in =-=[13]-=-. Dynamic range: We first consider the photodetector dynamic range required to match the performance of the baseline PA. If each detector in the PA has a linear dynamic range of 0 to D, then it is eas... |

3 |
Optical architectures for compresssive imaging
- Ke, Neifeld
- 2007
(Show Context)
Citation Context ...given light signal or the transform coefficients in a basis of interest. The elegant hardware designed for these purposes uses optical projections, group testing, and signal inference. Recent work in =-=[29]-=- has compared several single and multiple pixel imaging strategies of varying complexities; their simulation 11sresults for Poisson counting noise agree closely with those above. Finally, in [30] the ... |

2 |
The smashed filter: Compressive sensing for signal classification
- Davenport, Duarte, et al.
- 2007
(Show Context)
Citation Context ...on and then apply a standard matched filtering technique. In contrast, the smashed filter (for dimensionally reduced matched filter) performs all of its operations directly on the random measurements =-=[18]-=-. The two key elements of the smashed filter are the generalized likelihood ratio test and the concept of an image appearance manifold [20]. If the parametric transformation affecting the template ima... |

1 |
Compressive optical MONTAGE
- Brady, Feldman, et al.
- 2005
(Show Context)
Citation Context ...atterns, which enables simple demultiplexing, and randomized Hadamard patterns, which yield a uniform signal-to-noise ratio among the measurements. Other compressive cameras developed to date include =-=[27, 28]-=-, which employ optical elements to perform transform coding of multispectral images. These designs obtain sampled outputs that correspond to coded information of interest, such as the wavelength of a ... |