In this paper, we present a method that integrates cues from shading, shadow and specular reflections for estimating directional illumination in a textured scene. Texture poses a problem for lighting estimation, since texture edges can be mistaken for changes in illumination condition, and unknown variations in albedo make reflectance model fitting inpractical. Unlike previous works which all assume known or uniform reflectance, our method can deal with the effects of textures by capitalizing on physical consistencies that exist among the lighting cues. Since scene textures do not exhibit such coherence, we use this property to minimize the influence of texture on illumination direction estimation. For the recovered light source directions, a technique for estimating their intensities in the presence of texture is also proposed. 1

Most illuminant estimation algorithms work with the assumption of one specific type of the light source (e.g. point light source or directional light source). This assumption brings up two main limitations which significantly restrict the applicability of the algorithms: First, the knowledge about the type of the light source presented in the scene is needed a priori; second, it can not handle complex scenes where multiple different types of light sources co-exist. To overcome these limitations, we remove the assumption about the source type and develop a general light source model for all different types of light sources. Based on this general light source model, we propose a unified framework to estimate multiple illuminants of different types. Within the framework, we use an experiment setup where a calibration sphere with a specular surface is utilized to probe the scene illuminants and a novel ray tracing and matching algorithm is devised to estimate the light source parameters. Experiment results demonstrate the accuracy of our method on a variety of real images. 1

Abstract We propose a generative model based method for recovering both the shape and the reflectance of the surface(s) of a scene from multiple images, assuming that illumination conditions and cameras calibration are known in advance. Based on a variational framework and via gradient descents, the algorithm minimizes simultaneously and consistently a global cost functional with respect to both shape and reflectance. The motivations for our approach are threefold. (1) Contrary to previous works which mainly consider specific individual scenarios, our method applies indiscriminately to a number of classical scenarios; in particular it works for classical stereovision, multiview photometric stereo and multiview shape from shading. It works with changing as well as static illumination. (2) Our approach naturally combines stereo, silhouette and shading cues in a single framework. (3) Moreover, unlike most previous methods dealing with only Lambertian surfaces, the proposed

Abstract—We present a method for simultaneously estimating the illumination of a scene and the reflectance property of an object from single view images—a single image or a small number of images taken from the same viewpoint. We assume that the illumination consists of multiple point light sources, and the shape of the object is known. First, we represent the illumination on the surface of a unit sphere as a finite mixture of von Mises-Fisher distributions based on a novel spherical specular reflection model that well approximates the Torrance-Sparrow reflection model. Next, we estimate the parameters of this mixture model including the number of its component distributions and the standard deviation of them, which correspond to the number of light sources and the surface roughness, respectively. Finally, using these results as the initial estimates, we iteratively refine the estimates based on the original Torrance-Sparrow reflection model. The final estimates can be used to relight single-view images such as altering the intensities and directions of the individual light sources. The proposed method provides a unified framework based on directional statistics for simultaneously estimating the intensities and directions of an unknown number of light sources, as well as the specular reflection parameter of the object in the scene. Index Terms—Inverse rendering, von Mises-Fisher distribution, finite mixture distribution, EM algorithm. 1

Abstract In this paper, we present a complete framework for recovering an object shape, estimating its reflectance properties and light sources from a set of images. The whole process is performed automatically. We use the shape from silhouette approach proposed by R. Szeliski in [40] combined with image pixels for reconstructing a triangular mesh according to the marching cubes algorithm. A classification process identifies regions of the object having the same appearance. For each region, a single point or directional light source is detected. Therefore, we use specular lobes, lambertian regions of the surface or specular highlights seen on images. An identification method jointly (i) decides what light sources are actually significant and (ii) estimates diffuse and specular coefficients for a surface represented by the modified Phong model [25]. In order to validate our algorithm efficiency, we present a case study with various objects, light sources and surface properties. As shown in the results, our system proves accurate even for real objects images obtained with an inexpensive acquisition system. Keywords shape from   silhouette marching cubes multiple light sources   detection reflectance properties recovery 1

Abstract In this paper, we propose a new method for estimating jointly light sources and reflectance properties of an object seen through images. A classification process firstly identifies regions of the object having the same appearance. An identification method is then applied for jointly (i) deciding what light sources are actually significant and (ii) estimating diffuse and specular coefficients for the surface.

Most illuminant estimation algorithms worked on point light sources or directional light sources. Little attempt has been made, however, to estimate area light sources. In this paper, we present a novel scheme that estimates the size and location of multiple area light sources using a set of stereo images of a sphere with a shiny surface. The parameters of the area light source are estimated by a novel algorithm which minimizes the matching error between the corresponding specular patches. Experiments on real images show that our method is accurate and robust in estimating the parameters of the area light source.

Abstract. This paper introduces a novel method for recovering both the light directions and camera poses from a single sphere. Traditional methods for estimating light directions using spheres either assume both the radius and center of the sphere being known precisely, or they depend on multiple calibrated views to recover these parameters. It will be shown in this paper that the light directions can be uniquely determined from the specular highlights observed in a single view of a sphere without knowing or recovering the exact radius and center of the sphere. Besides, if the sphere is being observed by multiple cameras, its images will uniquely define the translation vector of each camera from a common world origin centered at the sphere center. It will be shown that the relative rotations between the cameras can be recovered using two or more light directions estimated from each view. Closed form solutions for recovering the light directions and camera poses are presented, and experimental results on both synthetic and real data show the practicality of the proposed method. 1

Recovering multiple point light sources from a sparse set of photographs in which objects of unknown texture can move is challenging. This is because both diffuse and specular reflections appear to slide across surfaces. What is seldom demonstrated, however, is that it can be taken advantage of to address the light-source recovery problem. In this paper, we therefore show that, if 3D models of the moving objects are available or can be computed from the images, we can solve the problem without any a priori constraints on the number of sources, on their color, or on the surface albedos. Our approach involves finding local maxima in individual images, checking them for consistency across images, retaining the apparently specular ones, and having them vote in a Hough-like scheme for potential light source directions. The precise directions of the sources and their relative power are then obtained by optimizing a standard lighting model. As a byproduct we also obtain an estimate of various material parameters such as the unlighted texture and specular properties. We show that the resulting algorithm can operate in presence of arbitrary textures and an unknown number of light sources of possibly different unknown colors. We also estimate its accuracy using ground-truth data. 1.

In this paper, we present a variational method that recovers both the shape and the reflectance of the Lambertian scene using multiple images. Although we consider only Lambertian surfaces in this paper, the proposed method, which is global and completely model based, is the first and unavoidable stage for reaching a shape and reflectance estimation method for non-Lambertian surfaces. Basically, our method is a multiview stereo/shape from shading algorithm which allows to recover 3D shapes from Lambertian shading with known illumination conditions. Contrary to previous works that deal with a single material object of the constant albedo, our method works for surfaces with non-constant reflectance parameters, in particular with non-constant albedo. In addition, our algorithm is not based on two or more separate steps – shape and reflectance are jointly recovered in a same process. We verified the proposed method using synthetic images. We will extend our method for non-Lambertian surfaces to improve the robustness to non-Lambertian effects. 1.