Abstract. We study the special form that the general multi-image tensor formalism takes under the plane + parallax decomposition, including matching tensors and constraints, closure and depth recovery relations, and inter-tensor consistency constraints. Plane + parallax alignment greatly simplifies the algebra, and uncovers the underlying geometric content. We relate plane + parallax to the geometry of translating, calibrated cameras, and introduce a new parallax-factorizing projective reconstruction method based on this. Initial plane + parallax alignment reduces the problem to a single rank-one factorization of a matrix of rescaled parallaxes into a vector of projection centres and a vector of projective heights above the reference plane. The method extends to 3D lines represented by viapoints and 3D planes represented by homographies.

Reconstruction of 3D structures from uncalibrated image sequences has a wealthy history. Most work has been focused on rigid objects or static scenes. This paper studies the problem of perspective reconstruction of deformable structures such as dynamic scenes from an uncalibrated image sequence. The task requires decomposing the image measurements into a composition of three factors: 3D deformable structures, rigid rotations and translations, and intrinsic camera parameters. We develop a factorization algorithm that consists of two steps. In the first step we recover the projective depths iteratively using the sub-space constraints embedded in the image measurements of the deformable structures. In the second step, we scale the image measurements by the reconstructed projective depths. We then extend the linear closed-form solution for weakperspective reconstruction [23] to factorize the scaled measurements and simultaneously reconstruct the deformable shapes and underlying shape model, the rigid motions, and the varying camera parameters such as focal lengths. The accuracy and robustness of the proposed method is demonstrated quantitatively on synthetic data and qualitatively on real image sequences. 1.

We develop a probabilistic framework for feature based multi-image matching that explicitly models the joint distribution of corresponding feature positions across several images. Conditioning this distribution on feature positions in some of the images gives welllocalized distributions for their correspondents in the others, which directly guide the correspondence search. This general framework is explored here in the simplest case of Gaussian distributions over the direct sum (affine images) and the tensor product (perspective images) of the image coordinates. Under these parametrizations, the formalism becomes a probabilistic generalization of the theory of multi-image matching constraints. It gracefully handles the full range of geometric correspondence models, including illconditioned near-planar ones intermediate between between full perspective and plane homographies. Small amounts of distortion and non-rigidity can also be tolerated. We develop the theory for any number of affi...

This paper presents a method for capturing and computing 3D parallax. 3D parallax, as used here, refers to vertical offset from the ground plane, height. The method is based on analyzing shadows of vertical poles (e.g., a tall building’s contour) that sweep the object. Unlike existing beam-scanning approaches, such as shadow or structured light, that recover the distance of a point from the camera, our approach measures the height from the ground plane directly. Previous methods compute the distance from the camera using triangulation between rays outgoing from the light-source and the camera. Such a triangulation is difficult when the objects are far from the camera, and requires accurate knowledge of the light source position. In contrast, our approach intersects two (unknown) planes generated separately by two casting objects. This omits the need to precompute the location of the light source. Furthermore, it allows a moving light source to be used. The proposed setup is particularly useful when the camera cannot directly face the scene or when the object is far away from the camera. A good example is an urban scene captured by a single webcam. 1

The views and conclusions contained in this document are those of the author and should not be interpreted as representing the official policies, either expressed or implied, of the Carnegie Mellon University or the U.S. Government or any of its agency. Biological or biomedical objects, such as expressive human faces and growing brain tumors, and dynamic scenes, such as cars running on the roads, generally vary their shapes as linear combinations of a number of shape bases. With the expeditious development of computer and imaging technologies, the problems of reconstruction, registration, and modeling of such deformable shapes from image measurements has shown enormous importance for applications such as biomedical image interpretation, human computer interaction, and robot navigation. Since the image measurements are generated by coupling two factors: non-rigid deformations and rigid similarity transformations between the shapes and the measurement systems, the essence of the three problems is to factorize the shape measurements and compute the deformable shapes (reconstruction), the rigid transformations (registration), and the shape bases

We propose a camera design where the camera records images through an attached aperture, which can be seen in the image. In such a design camera calibration is reduced to the problem of camera localization, with 3 degrees of freedom remaining which correspond to the unknown location of the camera. 9 degrees of freedom which correspond to the unknown camera's internal parameters and orientation are eliminated by the registration of the aperture. We discuss two applications: pointing target detection when pointing towards the camera, and depth estimation from two or more uncalibrated cameras. We conclude with experimental results which demonstrate the usefulness and robustness of our approach.