Shape from texture is best analyzed in two stages, analogous to stereopsis and structure from motion: (a) Computing the `texture distortion' from the image, and (b) Interpreting the `texture distortion' to infer the orientation and shape of the surface in the scene. We model the texture distortion for a given point and direction on the image plane as an affine transformation and derive the relationship between the parameters of this transformation and the shape parameters. We have developed a technique for estimating affine transforms between nearby image patches which is based on solving a system of linear constraints derived from a differential analysis. One need not explicitly identify texels or make restrictive assumptions about the nature of the texture such as isotropy. We use non-linear minimization of a least squares error criterion to recover the surface orientation (slant and tilt) and shape (principal curvatures and directions) based on the estimated affine transforms in a number of different directions. A simple linear algorithm based on singular value decomposition of the linear parts of the affine transforms provides the initial guess for the minimization procedure. Experimental results on both planar and curved surfaces under perspective projection demonstrate good estimates for both orientation and shape. A sensitivity analysis yields predictions for both computer vision algorithms and human perception of shape from texture.

This paper addresses the problem of computing cues to the three-dimensional structure of surfaces in the world directly from the local structure of the brightness pattern of either a single monocular image or a binocular image pair. It is shown that starting from Gaussian derivatives of order up to two at a range of scales in scale-space, local estimates of (i) surface orientation from monocular texture foreshortening, (ii) surface orientation from monocular texture gradients, and (iii) surface orientation from the binocular disparity gradient can be computed without iteration or search, and by using essentially the same basic mechanism. The methodology is based on a multi-scale descriptor of image structure called the windowed second moment matrix, which is computed with adaptive selection of both scale levels and spatial positions. Notably, this descriptor comprises two scale parameters; a local scale parameter describing the amount of smoothing used in derivative computations, and a...

Projective distortion of surface texture observed in a perspective image can provide direct information about the shape of the underlying surface. Previous theories have generally concerned planar surfaces; in this paper we present a systematic analysis of first- and second-order texture distortion cues for the case of a smooth curved surface. In particular, we analyze several kinds of texture gradients and relate them to surface orientation and surface curvature. The local estimates obtained from these cues can be integrated to obtain a global surface shape, and we show that the two surfaces resulting from the well-known tilt ambiguity in the local foreshortening cue typically have qualitatively different shapes. As an example of a practical application of the analysis, a shape from texture algorithm based on local orientation-selective filtering is described, and some experimental results are shown. i Figure 1: This image of a slanting plane covered with circles illustrates several...

: The problem of scale in shape from texture is addressed. The need for (at least) two scale parameters is emphasized; a local scale describing the amount of smoothing used for suppressing noise and irrelevant details when computing primitive texture descriptors from image data, and an integration scale describing the size of the region in space over which the statistics of the local descriptors is accumulated. A novel mechanism for automatic scale selection is proposed, based on normalized derivatives. It is used for adaptive determination of the two scale parameters in a multi-scale texture descriptor, the windowed second moment matrix, which is defined in terms of Gaussian smoothing, first order derivatives, and non-linear pointwise combinations of these. The same scale-selection method can be used for multi-scale blob detection without any tuning parameters or thresholding. The resulting texture description can be combined with various assumptions about surface texture in order to ...

This article describes a method for reducing the shape distortions due to scale-space smoothing that arise in the computation of 3-D shape cues using operators (derivatives) de ned from scale-space representation. More precisely, we are concerned with a general class of methods for deriving 3-D shape cues from 2-D image data based on the estimation of locally linearized deformations of brightness patterns. This class

Segmenting 3D textured surfaces is critical for general image understanding.

A neural model is presented of how cortical areas V1, V2, and V4 interact to convert a textured 2D image into a representation of curved 3D shape. Two basic problems are solved to achieve this: (1) Patterns of spatially discrete 2D texture elements are transformed into a spatially smooth surface representation of 3D shape. (2) Changes in the statistical properties of texture elements across space induce the perceived 3D shape of this surface representation. This is achieved in the model through multiple-scale filtering of a 2D image, followed by a cooperative-competitive grouping network that coherently binds texture elements into boundary webs at the appropriate depths using a scale-to-depth map and a subsequent depth competition stage. These boundary webs then gate filling-in of surface lightness signals in order to form a smooth 3D surface percept. The model quantitatively simulates challenging psychophysical data about perception of prolate ellipsoids [Todd, J., & Akerstrom, R. (1987). Perception of three-dimensional form from patterns of optical texture. Journal of Experimental Psychology: Human Perception and Performance, 13(2), 242–255]. In particular, the model represents a high degree of 3D curvature for a certain class of images, all of whose texture elements have the same degree of optical compression, in accordance with percepts of human observers. Simulations of 3D percepts of an elliptical cylinder, a slanted plane, and a photo of a golf ball are also presented.

this paper, these modified versions are basically the same as those described in [8, 1]. Other shape from texture methods (see [13]) have also been tested using other images, with similar results to those presented here. Detailed results are presented for method K&C. This method assumes that the texture on the surface is homogeneous in a sense which amounts informally to assuming that the distribution of surface texture in the image is the same as the distribution of surface area in the image.

Abstract not found

Image texture is useful for segmentation and for computing surface orientations of uniformly textured objects. If texture is ignored, it can cause failure for stereo and gray-scale segmentation algorithms. In the past, mathematical representations of image texture have been applied to only specific texture problems, and no consideration has been given to the models' generality across different computer vision tasks and different image phenomena. We advocate the space/frequency representation, which shows the local spatial frequency content of every point in the image. From several different methods of computing the representation, we pick the spectrogram. The spectrogram elucidates many disparate image phenomena including texture boundaries, texture in perspective, aliasing, zoom, and blur. Many past