Abstract-Several recently developed techniques for reconstructing surface shape from shading information estimate surface slopes with-out ensuring that they are integrable. This paper presents an approach for enforcing integrability, a particular implementation of the ap-proach, an example of its application to extending an existing shape-from-shading algorithm, and experimental results showing the im-provement that results from enforcing integrability. A possibly nonintegrable estimate of surface slopes is represented by a finite set of basis functions, and integrability is enforced by cal-culating the orthogonal projection onto a vector subspace spanning the set of integrable slopes. This projection maps closed convex sets into closed convex sets and, hence, is attractive as a constraint in iterative algorithms. The same technique is also useful for noniterative algo-rithms since it provides a least-squares fit of integrable slopes to non-integrable slopes in one pass of the algorithm. The special case of Fou-

Abstract: The method described here for recovering the shape of a surface from a shaded image can deal with complex, wrinkled surfaces. Integrability can be enforced easily because both surface height and gradient are represented (A gradient field is integrable if it is the gradient of some surface height function). The robustness of the method stems in part from linearization of the reflectance map about the current estimate of the surface orientation at each picture cell (The reflectance map gives the dependence of scene radiance on surface orientation). The new scheme can find an exact solution of a given shape-from-shading problem even though a regularizing term is included. The reason is that the penalty term is needed only to stabilize the iterative scheme when it is far from the correct solution; it can be turned off as the solution is approached. This is a reflection of the fact that shape-from-shading problems are not ill-posed when boundary conditions are available, or when the image contains singular points. This paper includes a review of previous work on shape from shading and photoclinometry. Novel features of the new scheme are introduced one at a time to make it easier to see what each contributes. Included is a discussion of implementation details that are important if exact algebraic solutions of synthetic shape-from-shading problems are to be obtained. The hope is that better performance on synthetic data will lead to better performance on real data.

The problem of using image contours to infer tile shapes and orientations of sur?aces is treated as a problem of statistical estimation. 'l'be basis for soMng this problem lies in an understanding of the geometry of contour fi)rmation, couplect with simple statistical models of the contour generating [,rocess. This approach is first applied to the special case of stu'faces known to be planar. The distortion of contour shape imposed by projection is treated as a signal to be estimated, and variations of non-projcctive origin are treated as noise. The resulting method is then extended to the estimation of curved surfaces, and applied successfidly to natural images. Next, the geometric tr.,atmcnt is furtber extended by relating contour curvature to surface curvature, using cast shadows as a model for contour generation. This geometric relation, combined with a statistical model, prox ides a measure of goodness-of-fit between a surface and an image contour. The goodnes-of-fit measure is applied to the problem of establishing registration between an image and a suffice nodel. Finally, the statistical estimation strategy is experimentally compared to human perception of orienbation: human observers' judgements of tilt correspond closely to the estimates produced by file planar strategy.

We present a system for fast model-based segmentation and 3D pose estimation of specular objects using appearance based specular features. We use observed (a) specular reflection and (b) specular flow as cues, which are matched against similar cues generated from a CAD model of the object in various poses. We avoid estimating 3D geometry or depths, which is difficult and unreliable for specular scenes. In the first method, the environment map of the scene is utilized to generate a database containing synthesized specular reflections of the object for densely sampled 3D poses. This database is compared with captured images of the scene at run time to locate and estimate the 3D pose of the object. In the second method, specular flows are generated for dense 3D poses as illumination invariant features and are matched to the specular flow of the scene. We incorporate several practical heuristics such as use of saturated/highlight pixels for fast matching and normal selection to minimize the effects of inter-reflections and cluttered backgrounds. Despite its simplicity, our approach is effective in scenes with multiple specular objects, partial occlusions, inter-reflections, cluttered backgrounds and changes in ambient illumination. Experimental results demonstrate the effectiveness of our method for various synthetic and real objects.

This paper introduces a novel surface model registration technique based on the Helmholtz reciprocity principle. Initially, a 3D surface model is generated for an object of interest. Two images of the object are then acquired under controlled lighting conditions. Given one of the images and an estimate of the surface pose, Helmholtz reciprocity is used to predict the appearance of the object surface as seen in the other image. This prediction is as accurate as one produced by a complete modeling of the BRDF of the surface, without requiring the BRDF to be explicitly measured. The position and orientation of the model are updated in order to minimize an appropriate metric of the dissimilarity between the predicted image and the observed second image. Experimental results are demonstrated for objects possessing a variety of surface properties. 1.

An algorithm is presented for video georegistration, with a particular concern for aerial video, i.e., video captured from an airborne platform. The algorithm's input is a video stream with telemetry (camera model specification sufficient to define an initial estimate of the view) and geodetically calibrated reference imagery (coaligned digital orthoimage and elevation map). The output is a spatial registration of the video to the reference so that it inherits the available geodetic coordinates. The video is processed in a continuous fashion to yield a corresponding stream of georegistered results. Quantitative results of evaluating the developed approach with real world aerial video also are presented. The results suggest that the developed approach may provide valuable input to the analysis and interpretation of aerial video.