## Relighting with the reflected irradiance field: Representation, sampling and reconstruction (2001)

### Cached

### Download Links

- [research.microsoft.com]
- [www.cse.cuhk.edu.hk]
- DBLP

### Other Repositories/Bibliography

Venue: | In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’01), Kauai |

Citations: | 25 - 2 self |

### BibTeX

@INPROCEEDINGS{Lin01relightingwith,

author = {Zhouchen Lin and Tien-tsin Wong},

title = {Relighting with the reflected irradiance field: Representation, sampling and reconstruction},

booktitle = {In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’01), Kauai},

year = {2001},

pages = {561--567}

}

### Years of Citing Articles

### OpenURL

### Abstract

Abstract. Image-based relighting (IBL) is a technique to change the illumination of an image-based object/scene. In this paper, we define a representation called the reflected irradiance field which records the light reflected from a scene as viewed at a fixed viewpoint as a result of moving a point light source on a plane. It synthesizes a novel image under a different illumination by interpolating and superimposing appropriate recorded samples. Furthermore, we study the minimum sampling problem of the reflected irradiance field, i.e., how many light source positions are needed. We find that there exists a geometry-independent bound for the sampling interval whenever the second-order derivatives of the surface BRDF and the minimum depth of the scene are bounded. This bound ensures that when the novel light source is on the plane, the error in the reconstructed image is controlled by a given tolerance, regardless of the geometry. We also analyze the bound of depth error so that the extra reconstruction error can also be governed when the novel light source is off-plane. Experiments on both synthetic and real surfaces are conducted to verify our analysis. Keywords: sampling, BRDF, light field, Lumigraph, plenoptic functions, image-based rendering, relighting

### Citations

2082 |
Matrix Computation
- Golub, Loan
- 1989
(Show Context)
Citation Context ...metry, instead of surface BRDF, by taking images of an object when a point light source moves on two concentric spheres enclosing the object. Unlike the above approaches, singular value decomposition =-=[14]-=- was applied to extract a set of basis images [43], where novel images were obtained by linearly combining the basis images. Unfortunately, most of the previous work ignored the error analysis of samp... |

1483 |
Robot Vision
- Horn
- 1986
(Show Context)
Citation Context ...xposure. Referring to Figure 5, the exposure at the pixel of A is: E = I i t; where I i is the irradiance received through the pixel window associated with A and t is the exposure time. As shown in =-=[18]-=-, I i is related to the intensity of the light source as follow: I i = cos 4 Ff 2 I r ; and I r = (` 1 ; ` 2 ; 1 ; 2 )` 2 d 2 I l ; where is the incident angle at C w.r.t. the optical axis ... |

1156 | Light field rendering
- Levoy, Hanrahan
- 1996
(Show Context)
Citation Context ...e can be of any kind, including directional light sources. Like IBMR, IBL is a problem of sampling and reconstructing the plenoptic function. Though for IBMR, the sampling problem of the light field (=-=Levoy and Hanrahan, 1996-=-; Gortler et al., 1996) has been studied by Lin and Shum (2000) and Chai et al. (2000), no previous work has addressed the sampling problem of IBL. Clearly, the sampling criteria of relighting and tha... |

905 | The lumigraph
- Gortler, Grzeszczuk, et al.
- 1996
(Show Context)
Citation Context ...uding directional light sources. Like IBMR, IBL is a problem of sampling and reconstructing the plenoptic function. Though for IBMR, the sampling problem of the light field (Levoy and Hanrahan, 1996; =-=Gortler et al., 1996-=-) has been studied by Lin and Shum (2000) and Chai et al. (2000), no previous work has addressed the sampling problem of IBL. Clearly, the sampling criteria of relighting and that of the light field a... |

702 | Plenoptic modeling: An image-based rendering system
- McMillan, Bishop
- 1995
(Show Context)
Citation Context ... pointed out by McMillan and Bishop (1995). To reduce the data volume, many researchers have worked on bringing down the dimensionality of the plenoptic function, ranging from five to two dimensions (=-=McMillan and Bishop, 1995-=-; Levoy and Hanrahan, 1996; Gortler et al., 1996; Shum and He, 1999; Chen, 1995). Although there are advantages to reducing the dimensionality of the plenoptic function, there are also reasons to incr... |

673 | Recovering high dynamic range radiance maps from photographs
- Debevec, Malik
- 1997
(Show Context)
Citation Context ...the interreflection between the object and the environment. Both the radiometric and geometric calibrations are necessary in real experiments. For radiometric calibration, we applied the algorithm in =-=[10]-=- to recover both the response function of the camera and the Special Issue on Multi-View Modeling and Rendering of Visual Scenes / IJCV 20 (c) (a) (b) (d) Figure 12: Elements of real experiments. (a) ... |

485 | The plenoptic function and the elements of early vision
- Adelson, Bergen
- 1991
(Show Context)
Citation Context ...IBMR (Image-based Modeling and Rendering) synthesizes images from pre-recorded images, which can be real photographs. It employs a collection of reference images as samples of the plenoptic function (=-=Adelson and Bergen, 1991-=-) so that the rendering process simply becomes ray query, interpolation and composition. Photorealistic images can be efficiently synthesized without a complex and long rendering process. Much of the ... |

478 | QuickTime VR – an image-based approach to virtual environment navigation
- Chen
- 1995
(Show Context)
Citation Context ... as pointed out by McMillan and Bishop [27]. To reduce the data volume, many researchers have worked on bringing down the dimensionality of the plenoptic function, ranging from five to two dimensions =-=[27, 22, 15, 36, 7]-=-. Special Issue on Multi-View Modeling and Rendering of Visual Scenes / IJCV 3 Although there are advantages to reducing the dimensionality of the plenoptic function, there are also reasons to increas... |

394 |
Distributed ray tracing
- Cook, Porter, et al.
- 1984
(Show Context)
Citation Context ...aptured by CCD cameras usually contain noise of around 9 greylevels. There also exists noise in synthetic images which are generated by stochastic rendering such as ray tracing with jittered samples (=-=Cook et al., 1984-=-). Moreover, the visual difference of a few greylevels is not apparent. Therefore, we suggest choosing a tolerance of about 16 greylevels. However, one may choose other error tolerance according to hi... |

384 |
Measuring and modeling anisotropic reflection
- Ward
- 1992
(Show Context)
Citation Context ...cause a wide range of reflectance models (both physical and empirical) satisfy this assumption, including the Lambertian model, the Lafortune model (Lafortune et al., 1997), Ward’s anisotropic model (=-=Ward, 1992-=-) and Oren and Nayar’s non-Lambertian model (Oren and Nayar, 1994). Moreover, an ordinary camera acts as a low-pass filter since its resolution is finite. Hence our assumption may be relaxed to those ... |

351 | Flexible camera calibration by viewing a plane from unknown orientations
- Zhang
- 1999
(Show Context)
Citation Context ... focal length because it will be cancelled during computation. For geometric calibration, we applied the toolbox in [5] to compute the intrinsic parameters and a standard camera calibration algorithm =-=[44]-=- to compute the homography between the image plane of the camera and the light source plane. Since the camera could not see both the light source and the object due to the limited FOV, during calibrat... |

343 | What is the set of images of an object under all possible lighting conditions
- Belhumeur, Kriegman
- 1996
(Show Context)
Citation Context ...ed an anti-aliasing algorithm that based on Phong model for specular highlight by clamping the parameters in the Phong model. Assuming that the object is convex and Lambertian, Belhumeur and Kriegman =-=[4]-=- have shown that three images are sufficient to relight the scene under various directional light sources. This paper is an extension of our previous work [24]. We propose the reflected irradiance fie... |

329 |
Acquiring the reflectance field of a human face
- Debevec, Hawkins, et al.
- 2000
(Show Context)
Citation Context ...inal lighting setup during the scene capture. Using higher dimensional plenoptic functions, Wong et al. [40] could both change the lighting (relight) and move the viewpoint. Similarly, Debevec et al. =-=[9]-=- also defined a variant of the extended plenoptic function called the reflectance field to represent the radiance reflected from human face where dense samples were captured and interpolation was igno... |

230 | Nonlinear Approximation of Reflectance Functions
- Lafortune, Foo, et al.
- 1997
(Show Context)
Citation Context ...tives are bounded. It is a reasonable assumption because a wide range of reflectance models (both physical and empirical) satisfy this assumption, including the Lambertian model, the Lafortune model (=-=Lafortune et al., 1997-=-), Ward’s anisotropic model (Ward, 1992) and Oren and Nayar’s non-Lambertian model (Oren and Nayar, 1994). Moreover, an ordinary camera acts as a low-pass filter since its resolution is finite. Hence ... |

223 | Rendering with concentric mosaics
- Shum, He
- 1999
(Show Context)
Citation Context ... researchers have worked on bringing down the dimensionality of the plenoptic function, ranging from five to two dimensions (McMillan and Bishop, 1995; Levoy and Hanrahan, 1996; Gortler et al., 1996; =-=Shum and He, 1999-=-; Chen, 1995). Although there are advantages to reducing the dimensionality of the plenoptic function, there are also reasons to increase it (Wong et al., to appear). Since the original formulation of... |

220 | Inverse global illumination : Recovering reflectance models of real scenes from photographs
- Yu, Debevec, et al.
(Show Context)
Citation Context ..., requires substantial knowledge of geometry and the rendering algorithms are generally complex (Shashua, 1997; Gibson and Murta, 2000; Loscos et al., 2000; Fournier et al., 1993; Yu and Malik, 1998; =-=Yu et al., 1999-=-; Sato et al., 1997), our analysis focuses on local reflection. The major assumption in our analysis is that the BRDF (ρ) in the scene must be second-order differentiable and the second-order derivati... |

208 | Plenoptic sampling
- Chai, Tong, et al.
- 2000
(Show Context)
Citation Context ...nd the center (lower curve) of the error map as a function of distance from the qr plane. Note that both curves reach their maximums around − 1 2 z∗ =−0.5. constant-depth plane (Isaksen et al., 2000; =-=Chai et al., 2000-=-; Lin and Shum, 2000). In the simplest case, the constant-depth plane is assumed to be the st plane. Hence the light gray pair of (q, r, s, t) is retrieved for interpolation, even though the correct o... |

200 | Reflection From Layered Surfaces Due To Subsurface Scattering
- Hanrahan, Krueger
- 1993
(Show Context)
Citation Context ...m 4.1. Background We assume a pin-hole camera, with a finite aperture to collect enough light. As the treatment of global illumination, such as cast shadow, interreflection and subsurface scattering (=-=Hanrahan and Krueger, 1993-=-), requires substantial knowledge of geometry and the rendering algorithms are generally complex (Shashua, 1997; Gibson and Murta, 2000; Loscos et al., 2000; Fournier et al., 1993; Yu and Malik, 1998;... |

196 | Object shape and reflectance modeling from observation
- Sato, Wheeler, et al.
- 1997
(Show Context)
Citation Context ...ntial knowledge of geometry and the rendering algorithms are generally complex (Shashua, 1997; Gibson and Murta, 2000; Loscos et al., 2000; Fournier et al., 1993; Yu and Malik, 1998; Yu et al., 1999; =-=Sato et al., 1997-=-), our analysis focuses on local reflection. The major assumption in our analysis is that the BRDF (ρ) in the scene must be second-order differentiable and the second-order derivatives are bounded. It... |

185 | Multiscale model of adaptation and spatial vision for realistic image display
- PATTANAIK, FERWERDA, et al.
- 1998
(Show Context)
Citation Context ...at is interesting and useful for future work is to determine the photometric tolerance which takes the response function of human vision into account. Recent work in psychophysical computer graphics (=-=Pattanaik et al., 1998-=-; Ferwerda et al., 1997) provides a psychophysical framework for measuring the sensitivity of human vision to artifacts due to undersampling. Another way to incorporate human perception into our sampl... |

155 | Dynamically reparameterized light fields
- Isaksen, McMillan, et al.
- 2000
(Show Context)
Citation Context ...ht end (upper curve) and the center (lower curve) of the error map as a function of distance from the qr plane. Note that both curves reach their maximums around − 1 2 z∗ =−0.5. constant-depth plane (=-=Isaksen et al., 2000-=-; Chai et al., 2000; Lin and Shum, 2000). In the simplest case, the constant-depth plane is assumed to be the st plane. Hence the light gray pair of (q, r, s, t) is retrieved for interpolation, even t... |

116 | Recovering photometric properties of architectural scenes from photographs
- Yu, Malik
- 1998
(Show Context)
Citation Context ...n and Krueger, 1993), requires substantial knowledge of geometry and the rendering algorithms are generally complex (Shashua, 1997; Gibson and Murta, 2000; Loscos et al., 2000; Fournier et al., 1993; =-=Yu and Malik, 1998-=-; Yu et al., 1999; Sato et al., 1997), our analysis focuses on local reflection. The major assumption in our analysis is that the BRDF (ρ) in the scene must be second-order differentiable and the seco... |

115 | On photometric issues in 3d visual recognition from a single 2d image
- Shashua
- 1997
(Show Context)
Citation Context ...lumination, such as cast shadow, interreflection and subsurface scattering (Hanrahan and Krueger, 1993), requires substantial knowledge of geometry and the rendering algorithms are generally complex (=-=Shashua, 1997-=-; Gibson and Murta, 2000; Loscos et al., 2000; Fournier et al., 1993; Yu and Malik, 1998; Yu et al., 1999; Sato et al., 1997), our analysis focuses on local reflection. The major assumption in our ana... |

106 | A model of visual masking for computer graphics
- Ferwerda, Pattanaik, et al.
- 1997
(Show Context)
Citation Context ...eful for future work is to determine the photometric tolerance which takes the response function of human vision into account. Recent work in psychophysical computer graphics (Pattanaik et al., 1998; =-=Ferwerda et al., 1997-=-) provides a psychophysical framework for measuring the sensitivity of human vision to artifacts due to undersampling. Another way to incorporate human perception into our sampling analysis is to anal... |

94 | Efficient re-rendering of naturally illuminated environments
- Nimeroff, Simoncelli, et al.
- 1994
(Show Context)
Citation Context ...k ignored the error analysis of sampling and reconstruction. Though the relit images were visually appealing, the correctness was never ensured. As an exception, superposition by steerable functions (=-=Nimeroff et al., 1994-=-) is exact for Lambertian surfaces, but it is limited to empirical cosine illumination models. The study on the related sampling issue is rare. Amanatides (1992) proposedRelighting with the Reflected... |

86 |
Lazy decompression of surface light fields for precomputed global illumination
- Miller, Rubin, et al.
- 1998
(Show Context)
Citation Context ...tured radiance values are rebinned so that all values associated to a pixel are grouped to form an image. Then, each of these images is compressed using 2D DCT. Block-coding technique is also used in =-=[28]-=-. Other coding techniques, such as vector quantization, and wavelet compression could be also used to reduce the data volume. However, random-access and selective decoding must be considered in select... |

84 | Generalization of lambert’s reflectance model
- Oren, Nayar
- 1994
(Show Context)
Citation Context ... and empirical) satisfy this assumption, including the Lambertian model, the Lafortune model (Lafortune et al., 1997), Ward’s anisotropic model (Ward, 1992) and Oren and Nayar’s non-Lambertian model (=-=Oren and Nayar, 1994-=-). Moreover, an ordinary camera acts as a low-pass filter since its resolution is finite. Hence our assumption may be relaxed to those BRDF models that are only first-order differentiable. To simplify... |

79 | Toward a psychophysically-based light reflection model for image synthesis
- Pellacini, Ferwerda, et al.
- 2000
(Show Context)
Citation Context ...n to artifacts due to undersampling. Another way to incorporate human perception into our sampling analysis is to analyze the bounds of the psychophysical reflection model that was recently proposed (=-=Pellacini et al., 2000-=-). In the current work, we compress the reflected irradiance field in the following manner. Firstly, the captured radiance values are rebinned so that all values associated to a pixel are grouped to f... |

69 |
Common Illumination between real and computer generated scenes
- Fournier, Gunavan, et al.
- 1993
(Show Context)
Citation Context ...ace scattering (Hanrahan and Krueger, 1993), requires substantial knowledge of geometry and the rendering algorithms are generally complex (Shashua, 1997; Gibson and Murta, 2000; Loscos et al., 2000; =-=Fournier et al., 1993-=-; Yu and Malik, 1998; Yu et al., 1999; Sato et al., 1997), our analysis focuses on local reflection. The major assumption in our analysis is that the BRDF (ρ) in the scene must be second-order differe... |

49 | Interactive Virtual Relighting of Real Scenes
- Loscos, Drettakis, et al.
(Show Context)
Citation Context ...eflection and subsurface scattering (Hanrahan and Krueger, 1993), requires substantial knowledge of geometry and the rendering algorithms are generally complex (Shashua, 1997; Gibson and Murta, 2000; =-=Loscos et al., 2000-=-; Fournier et al., 1993; Yu and Malik, 1998; Yu et al., 1999; Sato et al., 1997), our analysis focuses on local reflection. The major assumption in our analysis is that the BRDF (ρ) in the scene must ... |

49 | Image-based rendering with controllable illumination
- Wong, Heng, et al.
- 1997
(Show Context)
Citation Context ...arious directional light sources. This paper is an extension of our previous work (Lin et al., 2001). We propose the reflected irradiance field for IBL. Its concept is close to that of apparent BRDF (=-=Wong et al., 1997-=-) and the reflectance field (Debevec et al., 2000). The apparent BRDF and the reflectance field use a directional light source, while our reflected irradiance field adopts a point light source that mo... |

48 |
QuickTime VR—An Image-Based Approach to Virtual Environment Navigation
- Chen
- 1995
(Show Context)
Citation Context ...orked on bringing down the dimensionality of the plenoptic function, ranging from five to two dimensions (McMillan and Bishop, 1995; Levoy and Hanrahan, 1996; Gortler et al., 1996; Shum and He, 1999; =-=Chen, 1995-=-). Although there are advantages to reducing the dimensionality of the plenoptic function, there are also reasons to increase it (Wong et al., to appear). Since the original formulation of the plenopt... |

46 | Beyond lambert: Reconstructing surfaces with arbitrary brdfs
- Magda, Zickler, et al.
- 2001
(Show Context)
Citation Context ...eavored to recover the surface BRDF and approximate geometry. In [12, 25, 13], the shadows are more carefully treated, but only Lambertian BRDF is assumed in their global illumination computation. In =-=[26]-=-, Magda et al. tried to recover the object geometry, instead of surface BRDF, by taking images of an object when a point light source moves on two concentric spheres enclosing the object. Unlike the a... |

28 |
Camera calibration toolbox for Matlab. Available from: http://www.vision.caltech.edu/bouguetj/ calib_doc
- Bouguet
- 2004
(Show Context)
Citation Context ...e only need to record the aperture area and the exposure time, without measuring the focal length because it will be cancelled during computation. For geometric calibration, we applied the toolbox in =-=[5]-=- to compute the intrinsic parameters and a standard camera calibration algorithm [44] to compute the homography between the image plane of the camera and the light source plane. Since the camera could... |

28 | Interactive Rendering with RealWorld Illumination”, Eleventh Eurographics Workshop on Rendering
- Gibson, Murta
- 2000
(Show Context)
Citation Context ... human face where dense samples were captured and interpolation was ignored. Yu and Malik [42], Yu et al. [41] and Sato et al. [34] endeavored to recover the surface BRDF and approximate geometry. In =-=[12, 25, 13]-=-, the shadows are more carefully treated, but only Lambertian BRDF is assumed in their global illumination computation. In [26], Magda et al. tried to recover the object geometry, instead of surface B... |

16 | On the number of samples needed in light field rendering with constant-depth assumption
- Lin, Shum
- 2000
(Show Context)
Citation Context ...e are three sampling rates to consider, one for each plane. The sampling rate on st depends on the required resolution of the desired image. The tricky part is the sampling of uv and qr. Lin and Shum =-=[23]-=- and Chai et al. [6] have addressed the sampling issue on the uv plane for view interpolation. In this paper, we first address the question of how many samples are enough to avoid noticeable artifacts... |

15 | Modeling geometric structure and illumination variation of a scene from real images
- Zhang
- 1998
(Show Context)
Citation Context ... light source moves on two concentric spheres enclosing the object. Unlike the above approaches, singular value decomposition (Golub and van Loanc, 1989) was applied to extract a set of basis images (=-=Zhang, 1998-=-), where novel images were obtained by linearly combining the basis images. Unfortunately, most of the previous work ignored the error analysis of sampling and reconstruction. Though the relit images ... |

9 | Algorithms for the detection and elimination of specular aliasing
- Amanatides
- 1992
(Show Context)
Citation Context ...ion, superposition by steerable functions [29] is exact for Lambertian surfaces, but it is limited to empirical cosine illumination models. The study on the related sampling issue is rare. Amanatides =-=[3]-=- proposed an anti-aliasing algorithm that based on Phong model for specular highlight by clamping the parameters in the Phong model. Assuming that the object is convex and Lambertian, Belhumeur and Kr... |

9 | The plenoptic illumination function - Wong, Fu, et al. - 2002 |

7 | Synthetic lighting for photography. available on http://www.sgi.com/grafica/synth/index.html - Haeberli - 1992 |

1 |
Acquiring the reflectance field of a Lin, Wong and Shum human face
- Debevec, Hawkins, et al.
- 2000
(Show Context)
Citation Context ...s an extension of our previous work (Lin et al., 2001). We propose the reflected irradiance field for IBL. Its concept is close to that of apparent BRDF (Wong et al., 1997) and the reflectance field (=-=Debevec et al., 2000-=-). The apparent BRDF and the reflectance field use a directional light source, while our reflected irradiance field adopts a point light source that moves on a plane. Our rationale is that sampling wi... |

1 |
Numerical Approximation (in Chinese
- Wang
- 1999
(Show Context)
Citation Context ...nimum distance zmin (see Fig. 6) between the object and the qr plane. In the following, we show that for BRDFs with bounded second-order derivatives, the bound is positive. From interpolation theory (=-=Wang, 1999-=-), there exists ˜q ∈ [q1, q2], such that Therefore, λE(q1) + (1 − λ)E(q2) − E(q) = 1 2 (q − q1)(q2 − q) ∂2 E(˜q) ∂q 2 . |λE(q1) + (1 − λ)E(q2) − E(q)| ≤ 1 [ ] 2 (q − q1) + (q2 − q) c1 B2 = 2 2 1 8 (�q... |

1 | irrad demo (version 1.0). available at http:// www.cse.cuhk.edu.hk/∼ttwong/demo/irrad/irrad. html - Wong |

1 |
Chi-Wing Fu, Pheng-Ann Heng, and Chi-Sing Leung. The plenoptic illumination function
- Wong
(Show Context)
Citation Context ...cial Issue on Multi-View Modeling and Rendering of Visual Scenes / IJCV 3 Although there are advantages to reducing the dimensionality of the plenoptic function, there are also reasons to increase it =-=[39]-=-. Since the original formulation of the plenoptic function assumes that lighting is constant, the illumination of generated images is also fixed. This limitation hampers the further application of IBM... |