We investigate the feasibility of reconstructing an arbitrarily-shaped specular scene (refractive or mirror-like) from one or more viewpoints. By reducing shape recovery to the problem of reconstructing individual 3D light paths that cross the image plane, we obtain three key results. First, we show how to compute the depth map of a specular scene from a single viewpoint, when the scene redirects incoming light just once. Second, for scenes where incoming light undergoes two refractions or reflections, we show that three viewpoints are sufficient to enable reconstruction in the general case. Third, we show that it is impossible to reconstruct individual light paths when light is redirected more than twice. Our analysis assumes that, for every point on the image plane, we know at least one 3D point on its light path. This leads to reconstruction algorithms that rely on an “environment matting” procedure to establish pixel-to-point correspondences along a light path. Preliminary results for a variety of scenes (mirror, glass, etc) are also presented.

Abstract. We propose a new method named compressive structured light for recovering inhomogeneous participating media. Whereas conventional structured light methods emit coded light patterns onto the surface of an opaque object to establish correspondence for triangulation, compressive structured light projects patterns into a volume of participating medium to produce images which are integral measurements of the volume density along the line of sight. For a typical participating medium encountered in the real world, the integral nature of the acquired images enables the use of compressive sensing techniques that can recover the entire volume density from only a few measurements. This makes the acquisition process more efficient and enables reconstruction of dynamic volumetric phenomena. Moreover, our method requires the projection of multiplexed coded illumination, which has the added advantage of increasing the signal-to-noise ratio of the acquisition. Finally, we propose an iterative algorithm to correct for the attenuation of the participating medium during the reconstruction process. We show the effectiveness of our method with simulations as well as experiments on the volumetric recovery of multiple translucent layers, 3D point clouds etched in glass, and the dynamic process of milk drops dissolving in water. 1

of the 3D gradient field tomographically reconstructed from 16 cameras. Far right: volume rendering of the final refractive index field after

We present a new method for reconstructing the exterior surface of a complex transparent scene with inhomogeneous interior (e.g., multiple interfaces, reflective or painted interiors, etc). Our approach involves capturing images of the scene from one or more viewpoints while moving a proximal light source to a 2D or 3D set of positions. This gives a 2D (or 3D) dataset per pixel, called the scatter trace. The key idea of our approach is that even though light transport within a transparent scene’s interior can be exceedingly complex, the scatter trace of each pixel has a highlyconstrained geometry that (1) reveals the contribution of direct surface reflection, and (2) leads to a simple “scattertrace stereo ” algorithm for computing the local geometry of the exterior surface (depth and surface normals). We present 3D reconstruction results for a variety of scenes that exhibit complex light transport phenomena.

We introduce a new approach to capturing refraction in transparent media, which we call Light Field Background Oriented Schlieren Photography (LFBOS). By optically coding the locations and directions of light rays emerging from a light field probe, we can capture changes of the refractive index field between the probe and a camera or an observer. Rather than using complicated and expensive optical setups as in traditional Schlieren photography we employ commodity hardware; our prototype consists of a camera and a lenslet array. By carefully encoding the color and intensity variations of a 4D probe instead of a diffuse 2D background, we avoid expensive computational processing of the captured data, which is necessary for Background Oriented Schlieren imaging (BOS). We analyze the benefits and limitations of our approach and discuss application scenarios. 1.

Figure 1: Measurements of the refractive index field caused by rising hot air from a gas burner. Far left: magnitude of refraction for each pixel in one view. Center left: vector field of 2D displacements. Center: distortion measurements from one view are used as an environment matte to distort the background, data from another view is used to cast a caustic (shadowgraph). Center right: maximum intensity projection volume rendering of a 3D refractive index field recovered with tomography from 8 views. Far right: volume rendering of the 3D reconstruction. We present a technique for 2D imaging and 3D tomographic reconstruction of time-varying, inhomogeneous refractive index fields. Our method can be used to perform three-dimensional reconstruction of phenomena such as gas plumes or liquid mixing. We can also use the 2D imaging results of such time-varying phenomena to render environment mattes and caustics. To achieve these results, we improve a recent fluid imaging technique called Background Oriented Schlieren imaging, and develop a novel theory for tomographic reconstructions from Schlieren images based on first principles of optics. We demonstrate our approach with two different measurement setups, and discuss example applications such as measuring the heat and density distribution in gas flows. 1

Abstract. Obscure glass is textured glass designed to separate spaces and “obscure ” visibility between the spaces. Such glass is used to provide privacy while still allowing light to flow into a space, and is often found in homes and offices. We propose and explore the challenge of “seeing through ” obscure glass, using both optical and digital techniques. In some cases – such as when the textured surface is on the side of the observer – we find that simple household substances and cameras with small apertures enable a surprising level of visibility through the obscure glass. In other cases, where optical techniques are not usable, we find that we can model the action of obscure glass as convolution of spatially varying kernels and reconstruct an image of the scene on the opposite side of the obscure glass with surprising detail. 1

Abstract. We describe a single-shot method to differentiate unscattered and scattered components of light transmission through a heterogeneous translucent material. Directly-transmitted components travel in a straight line from the light source, while scattered components originate from multiple scattering centers in the volume. Computer vision methods deal with participating media via 2D contrast enhancing software techniques. On the other hand, optics techniques treat scattering as noise and use elaborate methods to reduce the scattering or its impact on the direct unscattered component. We observe the scattered component on its own provides useful information because the angular variation is low frequency. We propose a method to strategically capture angularly varying scattered light and compute the unscattered direct component. We capture the scattering from a single light source via a lenslet array placed close to the image plane. As an application, we demonstrate enhanced tomographic reconstruction of scattering objects using estimated direct transmission images.

for her never-ending love, support, and gentle guidance throughout the years. iii PREFACE ‘Determinism is the philosophical proposition that every event, including human cognition and behaviour, decision and action, is causally determined by an unbroken chain of prior occurrences...’- Peter Van Inwagen, in An Essay on Free Will The world is as precise as a clock. When I was learning physics for the first time as a kid, I was fascinated by how accurately this world can be predicted by all kinds of physical laws. If you know how fast you throw an apple and where it leaves your hand, you can predict whether it will hit Isaac Newton’s head. If you shine a laser pen towards mirrors and lenses, the optics provides you the whole light path so that you may defeat the entire Roman fleet, as Archimedes did. I could not stop imagining that one day everything can be predicted in this world, given all of physical laws. Obviously I was not the first one with this wild idea. Actually, philosophers already had a name for it: determinism. Soon I learned that quantum mechanics is another

In recent years there has been significant progress in increasing the scope, accuracy and flexibility of 3D photography methods. However there are still significant open problems where complex optical properties of mirroring or transparent objects cause many assumptions of traditional algorithms to break down. In this work we present three approaches that attempt to deal with some of these challenges using a few camera views and simple illumination. First, we consider the problem of reconstructing the 3D position and surface normal of points on a time-varying refractive surface. We show that two viewpoints are sufficient to solve this problem in the general case, even if the refractive index is unknown. We introduce a novel “stereo matching ” criterion called refractive disparity, appropriate for refractive scenes, and develop an optimization-based algorithm for individually reconstructing the position and normal of each point projecting to a pixel in the input views. Second, we present a new method for reconstructing the exterior surface of a complex transparent scene with inhomogeneous interior. We capture images from each viewpoint while ii moving a proximal light source to a 2D or 3D set of positions giving a 2D (or 3D) dataset per