When a Lambertian surface is illuminated by several chromatic lights the surface normals may berecovered from a single color image. A robust regression is used to #nd the ellipsoid in color space on which at least half the pixels lie. Then the matrix giving the linear relationship between the color and the surface normal, for non-outlier points, is found as a root of the ellipsoid quadratic form. But this root is recovered only up to an arbitrary rotation. An integrability condition can be used to determine the correct rotation. The rotation of recovered surface normals is needed to align partial derivatives p and q with the camera plane and thus establish the object's attitude. Here a new smoothness condition approximating the integrability condition is introduced that allows one to solve for the rotation matrix in closed form.

If a uniformly colored Lambertian surface is illuminated by a collection of point or extended light sources or interreflections, with unknown directions and strengths, such that illumination varies spectrally with orientation from the surface, then surface normals can be recovered up to an orthogonal transformation using a robust regression on points in color space. Recently, it was shown that the unknown orthogonal transformation can be recovered by applying an integrability condition on the recovered normals. However, the integrability method results in an unavoidable convex/concave ambiguity additional to the usual one. Here a much simpler method is set out that avoids this ambiguity. Using Pentland's or a similar tilt estimator for each of the RGB channels in turn, in effect treating the combination of lights as three single sources, the robust color space regression leads to three constraints on the slants of the three sources. The result is accurate recovery of light source direc...

Several color object recognition methods that are based on image retrieval algorithms attempt to discount changes of illumination in order to increase performance when test image illumination conditions differ from those that obtained when the image database was created. Here we investigate under what general conditions illumination change can be described using a simple linear transform among RGB channels, for a multi--colored object, and adduce a different underlying principle than that usually suggested. The resulting new method, the Linear Color algorithm, is more accurately illuminant--invariant than previous methods. An implementation of the method uses a combination of wavelet compression and DCT transform to fully exploit the technique of low--pass filtering for efficiency. Results are very encouraging, with substantially better performance than other methods tested. The method is also fast, in that the indexing process is entirely carried out in the compressed domain and uses a feature vector of only 63 integers. 1

Photometric stereo is a well--known technique for recovering surface normals of a surface but requires three or more images of a surface taken under illumination from different directions. At best, one may dispense with the need for multiple images by using colored lights tuned to camera filters. But a less restrictive paradigm is available using the Orientation--from--Color approach, wherein multiple broadband illuminants impinge on a surface simultaneously. In that method, colors for a Lambertian surface lie on an ellipsoid in color space. The method has mostly been applied to single--color objects, with ellipsoid quadratic form parameters determined from a large number of pixels. However, recently Petrov et al. developed an entirely local approach, useful also for multicolored objects with color uniform in each patch. Here we investigate to what extent a method like Petrov's can be applied in the ostensibly simpler situation in which the complex lighting environment is known, i.e. a color photometric stereo situation, with all lights at play at once with only a single image to analyze. We find that, assuming a simple model of color formation, we are able to recover the object colors along with surface normals, using only a single image. Because we immerse the object in a known lighting environment, we show that only half of the equations utilized by Petrov are actually needed, making the method more stable. Nevertheless solutions do not exist at every pixel; instead we may determine a best estimate of patch color using a robust estimator, and then apply that estimate throughout a patch. Results are shown to be quite good, compared to ground truth. The simple color model can often be made to hold more exactly by transforming the color space to one corresponding to spe...

Photometric Stereo (PMS) recovers orientation vectors from a set of graylevel images. Under orthography, when the lights are unknown, and for a single uniform Lambertian surface, one can recover surface normals up to an unknown overall orthogonal transformation. The same situation obtains if, instead of three graylevel images, one uses a single RGB image taken with at least three point or extended colored lights impinging on the surface at once. Then using a robust technique and the constraints among the resulting three effective lighting vectors one can recover effective lights as well as normals, with no unknown rotation. However, in the case of a non--Lambertian object, PMS reduces to the idea of using a lookup table (LUT) based on a calibration sphere. Here, we show that a LUT can also be used in the many--colored--lights paradigm, eliminating the need for three separate images as in standard PMS. As well, we show how to transform a calibration sphere made of a particular material ...

Flash/noflash pairs have been studied as a simple mechanism for supplying less-noisy information for dark areas of an ambient-light image. This research direction also includes image pairs or sequences of nighttime or obscured imagery such as surveillance video. Ambient imagery has some advantages over flash images, in general, in that a flash can create harsh or unpleasing effects, and can also generate additional shadowing. The extra shadows from the flash are in fact quite amenable to detection. However, filling in information in image areas occupied by shadows in the ambient image has not been effectively considered. Here we consider the problem of detecting shadows in the image taken under ambient light, given extra information from a flash image registered with the first. Clearly, the pure-flash image is the difference between the two — and has no ambient-shadow component. But the question of how best to detect the ambient shadow remains. We argue that first going to a “spectrally sharpened ” color space, and then focusing on the difference in a log domain of the flash image and the ambient image, gives a very simple feature space consisting of two components — one in an illuminant-change 3-vector direction, and one along the gray axis. This space provides excellent separation of the shadow and nonshadow areas. Regressing pixel data from the flash image to the nonflash one adjusts the scale properly, and then inserting edges from the flash image inside the ambient-shadow region into the ambient image edge map and inverting Poisson’s equation fills in the shadow. In this way, we arrive at an image with the advantages of the ambient-only image — warmth, no flash effects such as disturbing illumination dropoff with distance and pixel saturation etc. — but no shadows. 1.

The image recorded by a camera depends on three factors: The physical content of the scene, the illumination incident on the scene, and the characteristics of the camera. This leads to a problem for many applications where the main interest is in the physical content of the scene. Consider, for example, a computer

Abstract — Shadows bedevil multimedia applications, e.g. seeing into shadow regions in surveillance video. Model-based and non-model based, statistical methods, spatial, temporal, and invariant-based methods have been devised for combatting the shadow problem. Here we take a different approach, by attenuating the shadow by utilizing a second image under another illuminant to remove the effect of shadow-edges from an edge map of each frame. As a precursor step, we examine flash/noflash still image pairs. A flash image provides lessened shadows, but other shadows are produced. We can produce a flash-only (no ambient) image by subtracting the two images. but several artifacts remain. Instead, we have used the pure-flash image to detect the ambient shadows [ICME06]. However that method may fail when there are flash-shadows or specularities in the copy region. Here we manipulate the gradient field using a smoothing step including the directionality of edges near the shadow boundary, with improved results. I.