## What is the Set of Images of an Object Under All Possible Lighting Conditions (1996)

### Cached

### Download Links

- [www.cs.columbia.edu]
- [www.cs.columbia.edu]
- [www1.cs.columbia.edu]
- [www1.cs.columbia.edu]
- [www.cs.columbia.edu]
- [www.cs.columbia.edu]
- [www1.cs.columbia.edu]
- [www1.cs.columbia.edu]
- [www-cvr.ai.uiuc.edu]
- [www1.cs.columbia.edu]
- [www.cs.berkeley.edu]
- [vision.ucsd.edu]
- [www.eecs.berkeley.edu]
- [www.cnbc.cmu.edu]
- [www.cs.drexel.edu]
- [www1.cs.columbia.edu]

Venue: | IEEE CVPR |

Citations: | 322 - 27 self |

### BibTeX

@ARTICLE{Belhumeur96whatis,

author = {Peter N. Belhumeur and David J. Kriegmant},

title = {What is the Set of Images of an Object Under All Possible Lighting Conditions},

journal = {IEEE CVPR},

year = {1996},

pages = {270--277}

}

### Years of Citing Articles

### OpenURL

### Abstract

The appearance of a particular object depends on both the viewpoint from which it is observed and the light sources by which it is illuminated. If the appearance of two objects is never identical for any pose or lighting conditions, then- in theory- the objects can always be distinguished or recognized. The question arises: What is the set of images of an object under all lighting conditions and pose? In this paper, ive consider only the set of images of an object under variable allumination (including multiple, extended light sources and attached shadows). We prove that the set of n-pixel images of a convex object with a Lambertian reflectance function, illuminated by an arbitrary number of point light sources at infinity, forms a convex polyhedral cone in IR " and that the dimension of this illumination cone equals the number of distinct surface normals. Furthermore, we show that the cone for a particular object can be constructed from three properly chosen images. Finally, we prove that the set of n-pixel images of an object of any shape and with an arbitrary reflectance function, seen under all possi-ble illumination conditions, still forms a convex cone in Rn. Th.ese results immediately suggest certain approaches to object recognition. Throughout this paper, we ofler results demonstrating the empirical validity of the illumination cone representation. 1

### Citations

3268 | Convex Analysis
- Rockafellar
- 1970
(Show Context)
Citation Context ...th the non-negative orthant of IR n . 1 Lemma 1. The set of images L0 is a convex cone in IR n . (For the definition of convexity and the definition of a cone, see (Canon, Cullum Jr. and Polak, 1970; =-=Rockafellar, 1970-=-).) Proof: L0 = L∩{x | x ∈ IR n , with all components of x ≥ 0}. Both L and the positive orthant are convex and the intersection of two convex sets is convex. So it follows that L0 is convex. Because ... |

2793 | Eigenfaces for recognition
- Turk, Pentland
- 1991
(Show Context)
Citation Context ...arge set of the possible images of an object under pose and/or illumination variation (Murase and Nayar, 1995; Pentland, Moghaddam and Starner, 1994; Poggio and Sung, 1994; Sirovitch and Kirby, 1987; =-=Turk and Pentland, 1991-=-). These methods have gone a long way in demonstrating the advantages of using much richer descriptions than simply sparse features like edges and corners for recognition. Still, a drawback of these a... |

1503 | Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection
- Belhumeur, Hespanha, et al.
- 1997
(Show Context)
Citation Context ...e dimensional linear subspace (i.e. &) that was constructed for each person. Experimentally, recognition rates were perfect when there was only moderate variation in illumination with minor shadowing =-=[l]-=-. While the distance to illumination subspace algorithm performed impressively, the experiments also indicated that its performance deteriorates as the test images move further and further away from t... |

1403 |
Robot Vision
- HORN
- 1986
(Show Context)
Citation Context ...urface normals and albedo given by B, seen under illumination s, produces an image x given by the following equation x = max(Bs, 0), (1) where max(., 0) zeros all negative components of the vector Bs =-=[5]-=-. Note that the negative components of Bs correspond to the shadowed surface points and are sometimes called attached shadows [ll]. Also, note that convexity of the object's shape is needed to guarant... |

959 |
Visual Learning and Recognition of 3–D Objects from Appearance
- Murase, Nayar
- 1995
(Show Context)
Citation Context ...ixel images of an object under constant illumination, but over all possible poses, is at most six dimensional. Murase and Nayar take advantage of this structure when constructing appearance manifolds =-=[6]-=-. However, if the pose is held constant, the variability due to illumination may be much larger as the set of possible lighting conditions is infinite dimensional. Nonetheless, we will show that the s... |

638 | View-based and modular eigenspaces for face recognition - Pentland, Moghaddam, et al. - 1994 |

582 | Example-based learning for view-based human face detection
- Sung, Poggio
- 1998
(Show Context)
Citation Context ... model or learn a representation that captures a large set of the possible images of an object under pose and/or illumination variation (Murase and Nayar, 1995; Pentland, Moghaddam and Starner, 1994; =-=Poggio and Sung, 1994-=-; Sirovitch and Kirby, 1987; Turk and Pentland, 1991). These methods have gone a long way in demonstrating the advantages of using much richer descriptions than simply sparse features like edges and c... |

410 | Recognition by linear combinations of models - Ullman, Basri - 1991 |

347 |
Theory for off-specular reflection from roughened surfaces
- Torrance, Sparrow
- 1967
(Show Context)
Citation Context ... images. Yet, most objects are nonconvex in shape and have reflectance functions which can be better approximated by more sophisticated physical (Oren and Nayar, 1995; Tagare and de Figueiredo, 1993; =-=Torrance and Sparrow, 1967-=-) and phenomenological (Koenderink and van Doorn, 1996) models. The question again arises: What can we say about the set of images of an object with a nonconvex shape and a non-Lambertian reflectance ... |

271 | Face Recognition: The problem of compensating for changes in illuminations Direction - Adini, Moses, et al. - 1997 |

127 |
A low-dimensional representation of human faces for arbitrary lighting conditions
- Hallinan
- 1994
(Show Context)
Citation Context ...truct the illumination subspace C of the face. The bottom half of the figure shows three basis images, lying in CO, that span the illumination subspace ,C for the face. from the Harvard Face Database =-=[4]-=-, a collection of images of faces seen under a range of lighting directions. For the purpose of this demonstration, we used the images of one person, taking six images from the set CO and using singul... |

121 | B.V.,“Spectral sharpening: sensor transformations for improved color constancy - Finlayson, Drew, et al. - 1994 |

116 | A.: The bas-relief ambiguity - Belhumeur, Kriegman, et al. - 1999 |

99 | Generalization of the lambertian model and implications for machine vision,” IJCV
- Oren, Nayar
- 1995
(Show Context)
Citation Context ...umination cone can be constructed from as few as three images. Yet, most objects are nonconvex in shape and have reflectance functions which can be better approximated by more sophisticated physical (=-=Oren and Nayar, 1995-=-; Tagare and de Figueiredo, 1993; Torrance and Sparrow, 1967) and phenomenological (Koenderink and van Doorn, 1996) models. The question again arises: What can we say about the set of images of an obj... |

84 |
Shape from interreflections
- Nayar, Ikeuchi, et al.
- 1991
(Show Context)
Citation Context ... be formed without interreflection x by x' = (I - PK)-lx where I is the identity matrix, P is a diagonal matrix with Pz,i equal to the albedo of pixel i, and I< is known as the interreflection kernel =-=[7]-=-. When there is no shadowing, all images lie in a 3-D linear space that would be generated from (1) by a pseudosurface whose normals and albedo B' are given by B' = (I- PA')-'B [7, 81. From Theorem 4,... |

79 | Body plans
- Forsyth, Fleck
- 1997
(Show Context)
Citation Context ...ision breaking the object down into sub regions and building “illumination subcones.” These illumination subcones could then be glued together in a manner similar to that of the “Body Plans” work of (=-=Forsyth and Fleck, 1997-=-). 2. The Illumination Cone In this section, we develop the illumination cone representation. To start, we make two simplifying assumptions: first, we assume that the surfaces of objects have Lamberti... |

73 | Geometry and Photometry in 3D Visual Recognition
- Shashua
- 1992
(Show Context)
Citation Context ...ere max(·, 0) zeros all negative components of the vector Bs (Horn, 1986). Note that the negative components of Bs correspond to the shadowed surface points and are sometimes called attached shadows (=-=Shashua, 1992-=-). Also, note that we have assumed that the object’s shape is convex at this point to avoid cast shadows, i.e. shadows that the object casts on itself. If the object is illuminated by k point light so... |

69 | Bidirectional Reflection Distribution Function Expressed in Terms of Surface Scattering Modes. ECCV - Koenderink, Doorn - 1996 |

64 |
Photometric stereo under a light-source with arbitrary motion,” JOSA
- Hayakawa
- 1994
(Show Context)
Citation Context ...ite number of point light sources (i.e., the sum becomes an integral). The product of B with all possible light source directions and strengths sweeps out a subspace in the n-dimensional image space (=-=Hayakawa, 1994-=-; Nayar and Murase, 1996; Shashua, 1992); we call the subspace created by B the illumination subspace L, where L = {x | x = Bs, ∀s ∈ IR 3 }. Note that the dimension of L equals the rank of B. Since B ... |

63 |
5+/-2 eigenimages suffice: An empirical investigation of low-dimensional lighting models
- Epstein, Hallinan, et al.
- 1995
(Show Context)
Citation Context ...ns: Can the cone be learned from as few as three images? The second assumption was needed so that we could ignore the effects of specularities. Yet, as the recent work of Yuille and Epstein has shown =-=[3]-=-, even for surfaces which appear to be highly specular, the Lambertian component of the reflectance function still dominates. So again, we believe that for most objects the illumination cone is a fair... |

59 | fitting with missing data: Applications to structure-from-motion and to characterizing intensity images - Linear - 1997 |

38 |
Reflections on Shading
- Forsyth, Zisserman
(Show Context)
Citation Context ...cognition algorithms. 5.1. Interreflection A surface is not just illuminated by the light sources but also through interreflections from points on the surface itself (Bajcsy, Lee and Leonardis, 1996; =-=Forsyth and Zisserman, 1991-=-). For a Lambertian surface, the image with interreflection x ′ is related to the image that would be formed without interreflection x by x ′ =(I−RK) −1 x where I is the identity matrix, R is a diagon... |

33 |
Low-dimensional procedure for the characterization of human faces
- Sirovitch, Kirby
- 1987
(Show Context)
Citation Context ...sentation that captures a large set of the possible images of an object under pose and/or illumination variation (Murase and Nayar, 1995; Pentland, Moghaddam and Starner, 1994; Poggio and Sung, 1994; =-=Sirovitch and Kirby, 1987-=-; Turk and Pentland, 1991). These methods have gone a long way in demonstrating the advantages of using much richer descriptions than simply sparse features like edges and corners for recognition. Sti... |

28 | Detection of diffuse and specular interface reflections and inter-reflections by color image segmentation - Bajcsy, Lee, et al. - 1996 |

23 | Geometry and Photometry
- Shashua
- 1992
(Show Context)
Citation Context ...0), (1) where max(., 0) zeros all negative components of the vector Bs [5]. Note that the negative components of Bs correspond to the shadowed surface points and are sometimes called attached shadows =-=[ll]-=-. Also, note that convexity of the object's shape is needed to guarantee that the object does not cast shadows on itself. If the object is seen under illumination by IC point light sources at infinity... |

18 | Dimensionlity of illumination in appearance matching
- Nayar, Murase
- 1996
(Show Context)
Citation Context ...ucting an illumination cone for each pose? Currently, Nayar and Murase are extending their appearance manifold representation by modeling illumination variation for each pose as a 3-D linear subspace =-=[8]-=-. However, this representation does not account for shadowing. 4.6 Object Recognition Ultimately, we intend to apply the illumination cone concept to recognition. In earlier face recognition work, we ... |

17 | fitting with missing data: Applications to structure from motion and characterizing intensity images - Linear - 1997 |

14 | Image segmentation and reflection analysis through color - Klinker, Shafer, et al. |

10 | A framework for the construction of reflectance maps for machine vision - Tagare, DeFigueiredo - 1993 |

5 | Learning and recognizing objects using illumination subspaces - Epstein, Yuille, et al. - 1996 |

4 |
Generic surface interpretation: Observability model
- Binford
- 1987
(Show Context)
Citation Context ... albedo on the surface of the object or discontinuities in albedo across the boundary of the object generate edges in images, these edges tend to be insensitive to a range of illumination conditions (=-=Binford, 1987-=-). Yet, edges do not contain all of the information useful for recognition. Furthermore, objects which are not simple polyhedra or are not composed of piecewise constant albedo patterns often produce ... |

4 |
A ray-based computational model of light sources and illumination
- Langer, Zucker
- 1995
(Show Context)
Citation Context ...ination may be much larger as the set of possible lighting conditions is infinite dimensional. Arbitrary illumination can be modeled as a scalar function on a four-dimensional manifold of light rays (=-=Langer and Zucker, 1995-=-). However, without limiting assumptions about the possible light sources, the bidirectional reflectance density functions, or object geometry, it is difficult to draw limiting conclusions about the s... |

3 | Applied Optimal Control - Jr, Ho |