## Sampling—50 years after Shannon (2000)

Venue: | Proceedings of the IEEE |

Citations: | 211 - 22 self |

### BibTeX

@INPROCEEDINGS{Unser00sampling—50years,

author = {Michael Unser},

title = {Sampling—50 years after Shannon},

booktitle = {Proceedings of the IEEE},

year = {2000},

pages = {569--587}

}

### Years of Citing Articles

### OpenURL

### Abstract

This paper presents an account of the current state of sampling, 50 years after Shannon’s formulation of the sampling theorem. The emphasis is on regular sampling, where the grid is uniform. This topic has benefited from a strong research revival during the past few years, thanks in part to the mathematical connections that were made with wavelet theory. To introduce the reader to the modern, Hilbert-space formulation, we reinterpret Shannon’s sampling procedure as an orthogonal projection onto the subspace of band-limited functions. We then extend the standard sampling paradigm for a representation of functions in the more general class of “shift-invariant” functions spaces, including splines and wavelets. Practically, this allows for simpler—and possibly more realistic—interpolation models, which can be used in conjunction with a much wider class of (anti-aliasing) prefilters that are not necessarily ideal low-pass. We summarize and discuss the results available for the determination of the approximation error and of the sampling rate when the input of the system is essentially arbitrary; e.g., nonbandlimited. We also review variations of sampling that can be understood from the same unifying perspective. These include wavelets, multiwavelets, Papoulis generalized sampling, finite elements, and frames. Irregular sampling and radial basis functions are briefly mentioned. Keywords—Band-limited functions, Hilbert spaces, interpolation, least squares approximation, projection operators, sampling,

### Citations

2075 |
A wavelet Tour of Signal Processing
- Mallat
(Show Context)
Citation Context ...polating signal values. As additional examples of admissible generating functions, we may consider any scaling function (to be defined below) that appears in the theory of the wavelet transform [35], =-=[79]-=-, [115], [139]. It is important to note, however, that scaling functions, which are often also denoted by , satisfy an additional two-scale relation, which is not required here but not detrimental eit... |

1732 |
Ten lectures on wavelets
- Daubechies
- 1992
(Show Context)
Citation Context ...immediate relevance to signal processing and communications. Recently, there has been strong revival of the subject, which was motivated by the intense activity taking place around wavelets (see [7], =-=[35]-=-, [80], and [85]). It soon became clear that the mathematics of wavelets were also applicable to sampling, but with more freedom because no multiresolution is required. This led researchers to reexami... |

1599 | Orthonormal bases of compactly supported wavelets
- Daubechies
- 1988
(Show Context)
Citation Context ...n (18), except that (47) includes an additional summation over the scale index . The wavelet expansion works because the analysis and synthesis wavelets and generate a biorthogonal basis of such that =-=[32]-=-. While a detailed discussion of wavelets is beyond the scope of this paper, we want to point out that the analysis tools and mathematics are essentially the same as those used in the modern formulati... |

547 |
Communication in the presence of noise
- Shannon
- 1949
(Show Context)
Citation Context ..., sampling, sampling theorem, Shannon, splines, wavelets. I. INTRODUCTION In 1949, Shannon published the paper “Communication in the Presence of Noise,” which set the foundation of information theory =-=[105]-=-, [106]. This paper is a masterpiece; both in terms of achievement and conciseness. It is undoubtedly one of the theoretical works that has had the greatest impact on modern electrical engineering [14... |

476 |
Wavelets and Subband Coding
- Vetterli, Kovaevi
- 1995
(Show Context)
Citation Context ...al values. As additional examples of admissible generating functions, we may consider any scaling function (to be defined below) that appears in the theory of the wavelet transform [35], [79], [115], =-=[139]-=-. It is important to note, however, that scaling functions, which are often also denoted by , satisfy an additional two-scale relation, which is not required here but not detrimental either. Specifica... |

357 |
Spline Functions Basic Theory
- Schumaker
- 1981
(Show Context)
Citation Context ...measuring aliasing. 2) Nonuniform Splines and Radial Basis Functions: Another fruitful approach to irregular sampling is to use specially tailored basis functions such as the nonuniform splines [37], =-=[100]-=-. The B-spline formalizm, in particular, is well suited for practical applications: the underlying B-spline basis functions are compactly supported and can be constructed systematically using de Boor’... |

273 | Interpolation of scattered data: distance matrices and conditionally positive definite functions, Constructive Approximation
- Micchelli
(Show Context)
Citation Context ...lli has proven that the radial basis function interpolation problem with an arbitrary set of sampling points has a unique solution for a relatively wide class of positive, increasing radial functions =-=[86]-=-. At first sight, the representation (51) looks rather similar to (9) for it also involves the shifts of a single template. However, the nature of the basis functions is fundamentally different. In th... |

257 | Cubic convolution interpolation for digital image processing
- Keys
- 1981
(Show Context)
Citation Context ... dB per decade, which is consistent with (45). The next graph in Fig. 10 provides a performance comparison of four piecewise cubic polynomial kernels of same support : a) Keys’ interpolation function =-=[66]-=-, which is frequently used for image interpolation, b) a cubic Lagrange-like interpolation [97], c) the cubic B-spline [59], [128], and d) the optimal O-Moms function recently derived in [18]. The las... |

256 |
A class of nonharmonic Fourier series
- Duffin, Schaeer
- 1952
(Show Context)
Citation Context ...most of the results presented in Section IV are also available for the multifunction case [19]. D. Frames The notion of frame, which generalizes that of a basis, was introduced by Duffin and Schaffer =-=[47]-=-; it plays a crucial role in nonuniform sampling [12]. A frame is essentially a set of functions that span the whole space, but are not necessarily linearly independent [3]. To be more precise, a sequ... |

247 |
Splines minimizing rotation-invariant semi-norms in sobolev spaces
- Duchon
- 1977
(Show Context)
Citation Context ...er dimensions, the B-spline formalism is no longer applicable unless the grid is separable. A more general approach is to use radial basis functions [93], which are closely related to splines as well =-=[46]-=-. The radial basis function model has the form (51) where the basis functions are centered on the sampling points ; they are radially symmetric because they depend on the distance only. Typical exampl... |

221 | An Introduction to Harmonic Analysis
- Katznelson
(Show Context)
Citation Context ... Summation Formula The standard form of Poisson’s summation formula is (see [117]) (A.6) where denotes the Fourier transform of the continuous time function . The reader is referred to [13], [16], or =-=[65]-=- for a rigorous mathematical treatment. Considering the function , the Fourier transform of which is (modulation property), we get (A.7) This is precisely the discrete-time Fourier transform of the se... |

198 |
Finite element analysis
- Szabó, Babuska
- 1991
(Show Context)
Citation Context ...the basis functions are typically chosen as short as possible and with minimal overlap, to facilitate the inversion of the matrices involved in the numerical solution of differential equations [114], =-=[118]-=-. In this area of application, the approximation order of the representation is the key parameter and the ’s do not need to be particularly smooth [113]. In the more recent multiwavelet constructions,... |

173 |
Certain Topics in Telegraph Transmission Theory
- Nyquist
- 1928
(Show Context)
Citation Context ... . Formula (1) is exact if is bandlimited to ; this upper limit is the Nyquist frequency, a term that was coined by Shannon in recognition of Nyquist’s important contributions in communication theory =-=[88]-=-. In the mathematical literature, (1) is known as the cardinal series expansion; it is often attributed to Whittaker in 1915 [26], [143] but has also been traced back much further [14], [58]. Shannon’... |

166 |
The Theory of Radial Basis Function Approximation
- Powell
- 1992
(Show Context)
Citation Context ... order of the spline goes to infinity [78]. In higher dimensions, the B-spline formalism is no longer applicable unless the grid is separable. A more general approach is to use radial basis functions =-=[93]-=-, which are closely related to splines as well [46]. The radial basis function model has the form (51) where the basis functions are centered on the sampling points ; they are radially symmetric becau... |

163 |
Symmetric iterative interpolation processes
- Deslauriers, Dubuc
- 1989
(Show Context)
Citation Context ...is commonly represented by the symbol . Interestingly, it has been shown that the interpolator and the orthonormal function associated with many families of function spaces (splines, Dubuc–Deslaurier =-=[42]-=-, etc.) both converge to Shannon’s sinc interpolator as the order (to be defined in Section IV-B) tends to infinity [9], [10], [130]. The sinc function is rather special in the sense that it is both i... |

163 |
Contribution to the problem of approximation of equidistant data by analytic functions
- Schoenberg
- 1946
(Show Context)
Citation Context ...is function with itself repeatedly, we construct the B-splines of degree , which are defined recursively as (16) These functions are known to generate the polynomial splines with equally spaced knots =-=[98]-=-, [99]. Specifically, if , then the signals defined by (9) are polynomials of degree within each interval ( odd), respectively, (1/2) (1/2) when is even, with pieces that are patched together such as ... |

152 |
A Fourier analysis of the finite element variational method. In: Constructive aspects of Functional Analysis. Edizione Cremonese
- Strang, Fix
- 1973
(Show Context)
Citation Context ...ty is to turn to approximation theory and to make use of the general error bounds that have been derived for similar problems, especially in connection with the finite element method [38]–[41], [73], =-=[113]-=-. Specialized error bounds have also been worked out for quasi-interpolation, which is an approximate form of interpolation without any prefilter [40], [74], [109], [110]. Unfortunately, these results... |

141 |
Fractal functions and wavelet expansions based on several scaling functions
- Geronimo, Hardin, et al.
- 1994
(Show Context)
Citation Context ... the more recent multiwavelet constructions, the multiscaling functions satisfy a vector two-scale relation—similar to (17)—that involves a matrix refinement filter instead of a scalar one [4], [45], =-=[51]-=-, [56], [116]. One of the primary motivation for this kind of extension is to enable the construction of scaling functions and wavelets that are symmetric (or antisymmetric), orthonormal, and compactl... |

128 | Approximation from shift-invariant subspaces of L2(R d - Boor, DeVore, et al. |

126 |
A General Sampling Theory for Nonideal Acquisition Devices
- Unser, Aldroubi
- 1994
(Show Context)
Citation Context ...form (9) with synthesis function . The solution is to apply a suitable digital correction filter , as shown in the block diagram in Fig. 4. Here, we consider a design based on the idea of consistency =-=[127]-=-. Specifically, one seeks a signal approximation that is such that it would yield exactly the same measurements if it was reinjected into the system. This is a reasonable requirement, especially when ... |

114 |
Cubic splines for image interpolation and digital filtering
- Hou, Andrews
- 1978
(Show Context)
Citation Context ...e cubic polynomial kernels of same support : a) Keys’ interpolation function [66], which is frequently used for image interpolation, b) a cubic Lagrange-like interpolation [97], c) the cubic B-spline =-=[59]-=-, [128], and d) the optimal O-Moms function recently derived in [18]. The last three generating functions are of order and are members of the so-called Moms (maximum order minimum support) family; the... |

108 |
Generalized sampling expansion
- Papoulis
- 1977
(Show Context)
Citation Context ...of Shannon’s sampling theory, showing that a band-limited signal could be reconstructed exactly from the samples of the response of linear shift-invariant systems sampled at 1 the reconstruction rate =-=[90]-=-. The main point is that there are many different ways of extracting information from a signal—a reconstruction is generally possible provided there are as many measurements as there are degrees of fr... |

105 |
On calculating with B-splines
- Deboor
- 1972
(Show Context)
Citation Context ...ne formalizm, in particular, is well suited for practical applications: the underlying B-spline basis functions are compactly supported and can be constructed systematically using de Boor’s recursion =-=[36]-=-, [37]. The expansion coefficients of the model are then determined from the solution of a sparse (band-diagonal) system of equations [37], [43]. One remarkable theoretical result, which makes the con... |

99 |
Necessary Density Conditions for Sampling and Interpolation of Certain Entire Functions.” Acta Mathematica 117
- Landau
- 1967
(Show Context)
Citation Context ...t spaces . 1) Irregular Sampling in Shift-Invariant Spaces: The problem that has been studied most extensively is the recovery of a band-limited function from its nonuniform samples [12], [48], [52], =-=[70]-=-, [96], [102]. A set for which a stable reconstruction is possible for all is called a set of sampling for . The stability requirement is important because there exist sets of samples that uniquely de... |

95 |
Comparison of interpolating methods for image resampling
- Parker, Troxel
- 1983
(Show Context)
Citation Context ... an ideal (anti-aliasing or reconstruction) low-pass filter. Third, Shannon’s reconstruction formula is rarely used in practice (especially with images) because of the slow decay of the sinc function =-=[91]-=-. Instead, practitioners typically rely on much simpler techniques such as PROCEEDINGS OF THE IEEE, VOL. 88, NO. 4, APRIL 2000 569 (1)sFig. 1. Frequency interpretation of the sampling theorem: (a) Fou... |

92 |
Feauveau. Biorthogonal bases of compactly supported wavelets
- Cohen, Daubechies, et al.
- 1990
(Show Context)
Citation Context ...n (18), except that (47) includes an additional summation over the scale index . The wavelet expansion works because the analysis and synthesis wavelets and generate a biorthogonal basis of such that =-=[32]-=-. While a detailed discussion of wavelets is beyond the scope of this paper, we want to point out that the analysis tools and mathematics are essentially the same as those used in the modern formulati... |

91 |
Fast b-spline transforms for continuous image representation and interpolation
- Unser, Aldroubi, et al.
- 1991
(Show Context)
Citation Context ...c polynomial kernels of same support : a) Keys’ interpolation function [66], which is frequently used for image interpolation, b) a cubic Lagrange-like interpolation [97], c) the cubic B-spline [59], =-=[128]-=-, and d) the optimal O-Moms function recently derived in [18]. The last three generating functions are of order and are members of the so-called Moms (maximum order minimum support) family; the cubic ... |

88 |
Multivariate interpolation at arbitrary points made simple
- Meinguet
- 1979
(Show Context)
Citation Context ...al extension of the cubic splines in multiples dimensions. The expansion coefficients in (51) are determined by solving a linear system of equations which expresses the interpolation constraint [28], =-=[84]-=-. Often, the solution includes an additional low-order polynomial term that is constrained to be orthogonal to the rest of the expansion. Micchelli has proven that the radial basis function interpolat... |

86 |
Improved resolution from sub-pixel shifted pictures,” CVGIP:Graphical Models and Image Processing
- Ur, Gross
- 1992
(Show Context)
Citation Context ...uction in the space of piecewise linear splines [61]. Recent applications of generalized sampling include motion-compensated deinterlacing of televison images [11], [121], and super-resolution [107], =-=[138]-=-. The latter is an attempt to reconstruct a higher resolution image from a series of lower resolution ones, which are shifted slightly with respect to each other. C. Finite Elements and Multiwavelets ... |

80 | Efficient numerical methods in nonuniform sampling theory
- Feichtinger, Gröchenig, et al.
- 1995
(Show Context)
Citation Context ...hat that a perfect reconstruction of a band-limited function from its nonuniform samples in is possible [17]. Efficient numerical methods for performing such reconstructions are described in [49] and =-=[50]-=-. More recently, researchers have extended these techniques to the more general wavelet and spline-like spaces [5], [29], [76], [77]. Aldroubi and Gröchenig derived generalized versions of the Beurlin... |

78 |
Massopust,\ Construction of orthogonal wavelets using fractal interpolation functions
- Donovan, Geronimo, et al.
- 1994
(Show Context)
Citation Context ...3]. In the more recent multiwavelet constructions, the multiscaling functions satisfy a vector two-scale relation—similar to (17)—that involves a matrix refinement filter instead of a scalar one [4], =-=[45]-=-, [51], [56], [116]. One of the primary motivation for this kind of extension is to enable the construction of scaling functions and wavelets that are symmetric (or antisymmetric), orthonormal, and co... |

68 | Approximation by translates of refinable functions
- Heil, Strang, et al.
- 1996
(Show Context)
Citation Context ...ore recent multiwavelet constructions, the multiscaling functions satisfy a vector two-scale relation—similar to (17)—that involves a matrix refinement filter instead of a scalar one [4], [45], [51], =-=[56]-=-, [116]. One of the primary motivation for this kind of extension is to enable the construction of scaling functions and wavelets that are symmetric (or antisymmetric), orthonormal, and compactly supp... |

64 | Surface interpolation with radial basis functions for medical imaging
- Carr, Fright, et al.
- 1997
(Show Context)
Citation Context ...iational extension of the cubic splines in multiples dimensions. The expansion coefficients in (51) are determined by solving a linear system of equations which expresses the interpolation constraint =-=[28]-=-, [84]. Often, the solution includes an additional low-order polynomial term that is constrained to be orthogonal to the rest of the expansion. Micchelli has proven that the radial basis function inte... |

64 |
On bandwidth
- Slepian
(Show Context)
Citation Context ...result is very elegant and has proven to be extremely fruitful, there are several problems associated with it. First, it is an idealization: real world signals or images are never exactly bandlimited =-=[108]-=-. Second, there is no such device as an ideal (anti-aliasing or reconstruction) low-pass filter. Third, Shannon’s reconstruction formula is rarely used in practice (especially with images) because of ... |

63 |
On the functions which are represented by the expansions of the interpolation theory
- Whittaker
- 1915
(Show Context)
Citation Context ...ion of Nyquist’s important contributions in communication theory [88]. In the mathematical literature, (1) is known as the cardinal series expansion; it is often attributed to Whittaker in 1915 [26], =-=[143]-=- but has also been traced back much further [14], [58]. Shannon’s sampling theorem and its corresponding reconstruction formula are best understood in the frequency domain, as illustrated in Fig. 1. A... |

61 |
The Shannon Sampling Theorem—Its Various Extensions and Applications: A Tutorial Review
- Jerri
(Show Context)
Citation Context ...ling theory, in its modern and extended versions, can perfectly handle such “nonideal” situations. Ten to 15 years ago, the subject of sampling had reached what seemed to be a very mature state [26], =-=[62]-=-. The research in this area had become very mathematically oriented, with less and less immediate relevance to signal processing and communications. Recently, there has been strong revival of the subj... |

60 |
Spline approximation by quasi-interpolants
- Boor, Fix
- 1973
(Show Context)
Citation Context ... with the finite element method [38]–[41], [73], [113]. Specialized error bounds have also been worked out for quasi-interpolation, which is an approximate form of interpolation without any prefilter =-=[40]-=-, [74], [109], [110]. Unfortunately, these results are mostly qualitative and not suitable for a precise determination of the approximation error. This has led researchers in signal processing, who wa... |

59 |
Approximation by multiinteger translates of functions having global support
- Jia, Lei
(Show Context)
Citation Context ...ing function, as proven by Xia [147]. Because of the importance of the finite elements in engineering, the quality of this type of approximation has been studied thoroughly by approximation theorists =-=[64]-=-, [73], [111]. In addition, most of the results presented in Section IV are also available for the multifunction case [19]. D. Frames The notion of frame, which generalizes that of a basis, was introd... |

58 |
Fractional splines and wavelets
- Unser, Blu
- 2000
(Show Context)
Citation Context ...ssible generating functions, which are not covered by the Strang–Fix theory, are the fractional B-splines of degree ; these satisfy the partition of unity but have a fractional order of approximation =-=[134]-=-. ACKNOWLEDGMENT The author wishes to thank A. Aldroubi and T. Blu for mathematical advice. He is also grateful to T. Blu, P. Thévenaz, and four anonymous reviewers for their constructive comments on ... |

56 |
Wavelets: A Mathematical Tool for Signal Analysis
- Chui
- 1997
(Show Context)
Citation Context ...cified by the uncertainty principle [129]. Furthermore, by using this type of basis function in a wavelet decomposition (see Section V-A), it is possible to trade one type of resolution for the other =-=[31]-=-, [129]. This is simply because the TFBP remains a constant irrespective of the scale of the basis function. F. Sampling and Reproducing Kernel Hilbert Spaces We now establish the connection between w... |

53 | Image interpolation and resampling
- Thévenaz, Blu, et al.
- 2000
(Show Context)
Citation Context ...s especially true in higher dimensions where the cost of prefiltering is negligible in comparison to the computation of the expansion formula. More details on the computational issues can be found in =-=[120]-=-. From the point of view of computation, the Shannon model has a serious handicap because there are no band-limited functions that are compactly supported—a consequence of the Paley–Wiener theorem [11... |

53 |
A sampling theorem for wavelet subspaces
- Walter
- 1992
(Show Context)
Citation Context ... allows for the representation of signals in terms of samples shifted by some fixed amount [60]. Walter has developed a similar representation in the more constrained setting of the wavelet transform =-=[141]-=-. E. Equivalent Basis Functions So far, we have encountered three types of basis functions: the generic ones ( ), the duals ( ), and the interpolating ones ( ). In fact, it is possible to construct ma... |

53 |
On nonuniform sampling of bandwidth-limited signals
- Yen
- 1956
(Show Context)
Citation Context ...cess is simplified: for example, in the Papoulis framework, it is achieved by multivariate filtering [23], [83]. Typical instances of generalized sampling are interlaced and derivative sampling [75], =-=[149]-=-, both of which are special cases of Papoulis’ formulation. While the generalized sampling concept is relatively straightforward, the reconstruction is not always feasible because of potential instabi... |

50 |
A Guide to Distribution Theory and Fourier Transforms, World Scientific
- Strichartz
- 2003
(Show Context)
Citation Context ...20]. From the point of view of computation, the Shannon model has a serious handicap because there are no band-limited functions that are compactly supported—a consequence of the Paley–Wiener theorem =-=[117]-=-. The use of windowed or truncated sinc functions is not recommended because these fail to satisfy the partition of unity; this has the disturbing consequence that the reconstruction error will UNSER:... |

46 |
Reconstruction algorithms in irregular sampling
- Gröchenig
- 1992
(Show Context)
Citation Context ...variant spaces . 1) Irregular Sampling in Shift-Invariant Spaces: The problem that has been studied most extensively is the recovery of a band-limited function from its nonuniform samples [12], [48], =-=[52]-=-, [70], [96], [102]. A set for which a stable reconstruction is possible for all is called a set of sampling for . The stability requirement is important because there exist sets of samples that uniqu... |

43 | Orthogonal multiwavelets with vanishing moments
- Strang, Strela
(Show Context)
Citation Context ...cent multiwavelet constructions, the multiscaling functions satisfy a vector two-scale relation—similar to (17)—that involves a matrix refinement filter instead of a scalar one [4], [45], [51], [56], =-=[116]-=-. One of the primary motivation for this kind of extension is to enable the construction of scaling functions and wavelets that are symmetric (or antisymmetric), orthonormal, and compactly supported. ... |

42 | H.: Exact iterative reconstruction algorithm for multivariate irregularly sampled functions in spline-like spaces: the L p -theory
- Aldroubi, Feichtinger
- 1998
(Show Context)
Citation Context ...ient numerical methods for performing such reconstructions are described in [49] and [50]. More recently, researchers have extended these techniques to the more general wavelet and spline-like spaces =-=[5]-=-, [29], [76], [77]. Aldroubi and Gröchenig derived generalized versions of the Beurling–Landau theorems based on an appropriate definition of the sampling density [6]. Specifically, they showed that t... |

42 |
Cardinal spline filters: Stability and convergence to the ideal sinc interpolator
- Aldroubi, Unser, et al.
- 1992
(Show Context)
Citation Context ...ero at all other integers. The interpolator for the space of cubic splines is shown in Fig. 6. It is rather similar to the sinc function, which is shown by the dotted line; for more details, refer to =-=[10]-=-. A slight modification of the scheme allows for the representation of signals in terms of samples shifted by some fixed amount [60]. Walter has developed a similar representation in the more constrai... |

36 | Controlled approximation and a characterization of the local approximation order
- Boor, Jia
- 1985
(Show Context)
Citation Context ...ne possibility is to turn to approximation theory and to make use of the general error bounds that have been derived for similar problems, especially in connection with the finite element method [38]–=-=[41]-=-, [73], [113]. Specialized error bounds have also been worked out for quasi-interpolation, which is an approximate form of interpolation without any prefilter [40], [74], [109], [110]. Unfortunately, ... |

34 |
Five short stories about the cardinal series
- Higgins
- 1985
(Show Context)
Citation Context ..., which gives a very comprehensive view of the subject up to the mid 1970’s [62]. Another useful source of information are the survey articles that appeared in the mathematical literature [26], [27], =-=[57]-=-. A. Wavelets In Section II, we have already encountered the scaling function , which plays a crucial role in wavelet theory. There, instead of a single space , one considers a whole ladder of rescale... |