## Video mosaics for virtual environments (1996)

Venue: | IEEE Computer Graphics and Applications |

Citations: | 276 - 12 self |

### BibTeX

@ARTICLE{Szeliski96videomosaics,

author = {Richard Szeliski and Microsoft Corporation},

title = {Video mosaics for virtual environments},

journal = {IEEE Computer Graphics and Applications},

year = {1996}

}

### Years of Citing Articles

### OpenURL

### Abstract

By panning a camera over a scene and automatically compositing the video frames, this system creates large panoramic images of arbitrary shape and detail. Depth recovery from motion parallax also enables limited 3D rendering. The use of photographic imagery as part of the computer graphics creation process is a well established and popular technique. Still imagery can be used in a variety of ways, including the manipulation and compositing of photographs inside video paint systems, and the texture mapping of still photographs onto 3D graphical models to achieve photorealism. Although laborious, it is also possible to merge 3D computer graphics seamlessly with video imagery to produce dramatic special effects. As computer-based video becomes ubiquitous with the expansion of transmission, storage, and manipulation capabilities, it will offer a rich source of imagery for computer graphics applications. This article looks at one way to use video as a new source of high-resolution, photorealistic imagery for these applications. In its current broadcast-standard forms, video is a low-resolution medium that compares poorly with computer displays and scanned imagery. It also suffers, as do all input imaging devices, from a limited field of view.

### Citations

1330 | Three-Dimensional Computer Vision - A Geometric Viewpoint - Faugeras - 1999 |

702 | Plenoptic modeling: An image-based rendering system
- McMillan, Bishop
- 1995
(Show Context)
Citation Context ...tial derivatives of and ei with respect to the m and t (which we concatenate into the motion vector m) as before in Equation 7. Similarly, we compute the partials of and with respect to wi . That is, =-=(12)-=- (13)swhere D i is the denominator in Equation 13. To estimate the unknown parameters, we alternate iterations of the Levenberg-Marquardt algorithm over the motion parameters {m0 , ..., t2 } and the d... |

550 | View interpolation for image synthesis
- Chen, Williams
- 1993
(Show Context)
Citation Context ...parameters {m0 , ..., t2 } and the depth parameters {wi }, using the partial derivatives defined above to compute the approximate Hessian matrices A and the weighted error vectors b as in Equation 8. =-=(14)-=- In the current implementation of this technique, the total number of parameters being estimated is decreased by using a tensor-product spline to represent the depth map and only recovering the depth ... |

408 |
Digital Image Warping
- Wolberg
- 1990
(Show Context)
Citation Context ...tical distortions that do not follow the pinhole model.) The combined equations projecting a 3D world coordinate p = (x, y, z, w) onto a 2D screen location u = (x', y', w') can thus be written as (2) =-=(3)-=-swhere P is a 3 × 4 camera matrix. This equation is valid even if the camera calibration parameters and/or the camera orientation are unknown. Planar image mosaics The simplest possible set of images ... |

378 |
Feature-Based Image Metamorphosis
- Beier, Shawn
- 1992
(Show Context)
Citation Context ...e optical distortions that do not follow the pinhole model.) The combined equations projecting a 3D world coordinate p = (x, y, z, w) onto a 2D screen location u = (x', y', w') can thus be written as =-=(2)-=- (3)swhere P is a 3 × 4 camera matrix. This equation is valid even if the camera calibration parameters and/or the camera orientation are unknown. Planar image mosaics The simplest possible set of ima... |

205 |
The phase correlation image alignment method
- Kuglin, Hines
- 1975
(Show Context)
Citation Context ...image intensity gradient of I’ at From these partial derivatives, the Levenberg-Marquardt algorithm computes an approximate Hessian matrix A and the weighted gradient vector b with components (5) (6) =-=(7)-=- (8)sand then updates the motion parameter estimate m by an amount m = (A + I) -1 b, where is a time-varying stabilization parameter. 5 The advantage of using Levenberg-Marquardt over straightforward ... |

179 |
Environment mapping and other applications of world projections
- Greene
- 1986
(Show Context)
Citation Context ... transformations have a general M matrix with 8 degrees of freedom. (Note that two M matrices are equivalent if they are scalar multiples of each other. We remove this redundancy by setting m 8 = 1.) =-=(1)-=-sFigure 1. Square and rigid, affine, and projective transformations. The same hierarchy of transformations exists in 3D, with rigid, similarity, affine, and full projective transformations having 6, 7... |

171 | Image mosaicing for tele-reality applications
- Szeliski
- 1994
(Show Context)
Citation Context ...a video camera mounted on a tripod, without taking any special steps to ensure that the rotation was around the true center of projection. As you can see, the complete scene is registered quite well. =-=(10)-=-sFigure 3. Panoramic image mosaic example (bookshelf and cluttered desk). These images were pasted onto a planar viewing surface. Another approach is to compute the relative position of each frame rel... |

144 | Hierarchical spline-based image registration
- Szeliski, Coughlan
- 1994
(Show Context)
Citation Context ...the image intensity gradient of I’ at From these partial derivatives, the Levenberg-Marquardt algorithm computes an approximate Hessian matrix A and the weighted gradient vector b with components (5) =-=(6)-=- (7) (8)sand then updates the motion parameter estimate m by an amount m = (A + I) -1 b, where is a time-varying stabilization parameter. 5 The advantage of using Levenberg-Marquardt over straightforw... |

121 | Virtual bellows: Constructing high-quality images from video
- Mann, Picard
- 1994
(Show Context)
Citation Context ... of the image. In practice, this approach completely eliminates edge artifacts (see Figure 2), although a low-frequency "mottling" might still remain if the individual tiles have different exposures. =-=(9)-=-sFigure 2. Whiteboard image mosaic example: (a) mosaic with component locations shown as colored outlines, (b) complete color mosaic (the central square shows the size of one input tile). Global image... |

109 |
et al, Numerical Recipes in C. The Art of Scientific Computing
- PRESS
- 1992
(Show Context)
Citation Context ... is the image intensity gradient of I’ at From these partial derivatives, the Levenberg-Marquardt algorithm computes an approximate Hessian matrix A and the weighted gradient vector b with components =-=(5)-=- (6) (7) (8)sand then updates the motion parameter estimate m by an amount m = (A + I) -1 b, where is a time-varying stabilization parameter. 5 The advantage of using Levenberg-Marquardt over straight... |

65 |
et al. Computer Graphics: Principles and Practice
- Foley
- 1990
(Show Context)
Citation Context ...entifiable feature points and of being statistically optimal, that is, giving the maximum likelihood estimate once we are in the vicinity of the true solution. Let's rewrite our 2D transformations as =-=(4)-=-sOur technique minimizes the sum of the squared intensity errors over all corresponding pairs of pixels i inside both images I(x, y) and I’(x’, y’). (Pixels that are mapped outside image boundaries do... |

48 |
QuickTime VR—An Image-Based Approach to Virtual Environment Navigation
- Chen
- 1995
(Show Context)
Citation Context ...anar viewing surface. Figure 6 shows a similar panorama taken on the banks of the Charles River in Cambridge. Figure 5. Circular panoramic image mosaic example (office interior). A total of 36 images =-=(11)-=-sare pasted onto a cylindrical viewing surface. Figure 6. Circular panoramic image mosaic example (exterior scene). A total of 29 images are pasted onto a cylindrical viewing surface. In addition to c... |

10 |
Pyramidal parametrics. Computer Graphics (Proc. Siggraph '83
- Williams
- 1983
(Show Context)
Citation Context ...int interpolation (using new M and matrices), or they can be converted to true Euclidean depth using at 7 least four known depth measurements. 15 < In more detail, we write the projection equation as =-=(13)-=- with We compute the partial derivatives of and e' i with respect to the m and t (which we concatenate into the motion vector m) as before in Equation 7. Similarly, we compute the partials of and with... |

7 |
Pyramidal Parametrics", Computer Graphics (SIGGRAPH '83
- Williams
- 1983
(Show Context)
Citation Context ...derivatives of and ei with respect to the m and t (which we concatenate into the motion vector m) as before in Equation 7. Similarly, we compute the partials of and with respect to wi . That is, (12) =-=(13)-=-swhere D i is the denominator in Equation 13. To estimate the unknown parameters, we alternate iterations of the Levenberg-Marquardt algorithm over the motion parameters {m0 , ..., t2 } and the depth ... |

7 | Panoramic overviews for navigating real-world scenes - Teodosio, Mills - 1993 |

5 |
Panoramic photographs
- Malde
- 1983
(Show Context)
Citation Context ...e intensity gradient of I’ at From these partial derivatives, the Levenberg-Marquardt algorithm computes an approximate Hessian matrix A and the weighted gradient vector b with components (5) (6) (7) =-=(8)-=-sand then updates the motion parameter estimate m by an amount m = (A + I) -1 b, where is a time-varying stabilization parameter. 5 The advantage of using Levenberg-Marquardt over straightforward grad... |