## Mercer’s Theorem, Feature Maps, and Smoothing

Citations: | 12 - 0 self |

### BibTeX

@MISC{Minh_mercer’stheorem,,

author = {Ha Quang Minh and Partha Niyogi and Yuan Yao},

title = {Mercer’s Theorem, Feature Maps, and Smoothing},

year = {}

}

### OpenURL

### Abstract

Abstract. We study Mercer’s theorem and feature maps for several positive definite kernels that are widely used in practice. The smoothing properties of these kernels will also be explored. 1

### Citations

2028 | Learning with Kernels
- Scholkopf, Smola
- 2002
(Show Context)
Citation Context ... and B1 3 (k + d + n − 2) 2d+n− 2 < λk < B2 3 (k + d + n − 2) d+n− 2 where 0 ≤ k ≤ d, for B1, B2 depending on d, n given below. Remark 2. B1 = e d (2e) d+n−2 d! 2.2 Example on the Hypercube {−1, 1} n =-=(7)-=- (8) n−1 n Γ (d+ 2 )Γ ( 2 ) 2π √ πe1/6d d+ 1 , B2 = e 2 d (2e) d+n−2 n−1 n Γ (d+ 2 )Γ ( 2 d! ) √ 2π We will now give an example with the hypercube {−1, 1} n . Let Mk = {α = (αi) n i=1 , αi ∈ {0, 1}, |... |

1273 |
Spline models for observational data
- Wahba
- 1990
(Show Context)
Citation Context ...e will show that for the polynomial and Gaussian kernels on S n−1 , they do. 5.1 The Iterated Laplacian and Splines on the Sphere S n−1 Splines on S n−1 for n = 2 and n = 3, as treated by Wahba [11], =-=[12]-=-, can be generalized to any n ≥ 2, n ∈ N, via the iterated Laplacian (also called the Laplace-Beltrami operator) on S n−1 . The RKHS corresponding to Wm in (20) is a subspace of L 2 (S n−1 ) described... |

777 |
Theory of reproducing kernels
- Aronszajn
- 1950
(Show Context)
Citation Context ...or each x ∈ X, let Kx : X → R be defined by Kx(t) = K(x, t) and HK = span{Kx : x ∈ X} (13) be the Reproducing Kernel Hilbert Space (RKHS) induced by K, with the inner product 〈Kx, Kt〉K = K(x, t), see =-=[1]-=-. The following feature map is then immediate: ΦK : X → HK ΦK(x) = Kx (14) In this section we discuss, via examples, two other methods for obtaining feature maps. Let X ⊂ R n be any subset. Consider t... |

330 | regularization: A geometric framework for learning from labeled and unlabeled examples
- Belkin, Niyogi, et al.
- 2006
(Show Context)
Citation Context ...t sphere S n−1 in R n , for several reasons. First, it is a special example of a compact Riemannian manifold and the problem of learning on manifolds has attracted attention recently, see for example =-=[2]-=-, [3]. Second, its symmetric and homogeneous nature allows us to obtain complete and explicit results in many cases. We believe that S n−1 together with kernels defined on it is a fruitful source of e... |

156 | Semi-supervised learning on Riemannian manifolds
- Belkin, Niyogi
- 2004
(Show Context)
Citation Context ...ere S n−1 in R n , for several reasons. First, it is a special example of a compact Riemannian manifold and the problem of learning on manifolds has attracted attention recently, see for example [2], =-=[3]-=-. Second, its symmetric and homogeneous nature allows us to obtain complete and explicit results in many cases. We believe that S n−1 together with kernels defined on it is a fruitful source of exampl... |

134 |
Theory of Bessel functions, 2nd edition
- Watson
- 1994
(Show Context)
Citation Context ... expression for λk as in (5). Lemma 1. Let f(t) = e rt , then � 1 −1 f(t)Pk(n; t)(1 − t 2 ) n−3 2 dt = √ πΓ ( n − 1 ) 2 � �n/2−1 2 Ik+n/2−1(r) (26) rsProof. We apply the following which follows from (=-=[13]-=-, page 79, formula 9) � �ν−1/2 2 Γ (ν)Iν−1/2(r) (27) r � 1 e −1 rt (1 − t 2 ) ν−1 dt = √ π and Rodrigues’ rule ([6], page 23), which states that for f ∈ C k ([−1, 1]) � 1 −1 f(t)Pk(n; t)(1 − t 2 ) n−3... |

86 | Diffusion kernels on statistical manifolds
- Lafferty, Lebanon
- 2005
(Show Context)
Citation Context ... √kθ }k∈N. Theorem 5 apπ π plies in particular to the Gaussian kernel K(x, t) = exp(− ||x−t||2 σ2 ). Hence care needs to be taken in applying analysis that requires CK = supk ||φk||∞ < ∞, for example =-=[5]-=-. 4 Feature Maps 4.1 Examples of Feature Maps via Mercer’s Theorem A natural feature map that arises immediately from Mercer’s theorem is Φµ : X → ℓ 2 Φµ(x) = ( � λkφk(x)) ∞ k=1 (12) where if only N <... |

64 |
Spline interpolation and smoothing on the sphere
- Wahba
- 1981
(Show Context)
Citation Context ...tion analyze the smoothing properties of the polynomial and Gaussian kernels and compare them with those of spline kernels on the sphere S n−1 . In the spline smoothing problem on S 1 as described in =-=[11]-=-, one solves the minimization problem 1 m m� (f(xi) − yi) 2 + λ i=1 � 2π 0 (f (m) (t)) 2 dt (20) for xi ∈ [0, 2π] and f ∈ Wm, where Jm(f) = � 2π 0 (f (m) (t)) 2 dt is the square norm of the RKHS W 0 m... |

40 | Learning theory estimates via integral operators and their approximations
- Smale, Zhou
- 2007
(Show Context)
Citation Context ...m, however, goes far beyond the feature maps that it induces: the eigenvalues and eigenfunctions associated with K play a central role in obtaining error estimates in learning theory, see for example =-=[8]-=-, [4]. For this reason, the determination of the spectrum of K, which is highly nontrivial in general, is crucially important in its own right. Theorems 2 and 3 in Section 2 give the complete spectrum... |

38 |
Analysis of spherical symmetries in Euclidean spaces, volume 129 of Applied Mathematical Sciences
- Müller
- 1998
(Show Context)
Citation Context ...ues and eigenfunctions in Mercer’s theorem on the unit sphere S n−1 for the polynomial and Gaussian kernels. We need the concept of spherical harmonics, a modern and authoritative account of which is =-=[6]-=-. Some of the material below was first reported in the kernel learning literature in [9], where the eigenvalues for the polynomial kernels with n = 3, were computed. In this section, we will carry out... |

38 |
The covering number in learning theory
- Zhou
(Show Context)
Citation Context ...s It is known that the L 2 µ-normalized eigenfunctions {φk} are generally unbounded, that is in general sup k∈N ||φk||∞ = ∞ This was first pointed out by Smale, with the first counterexample given in =-=[14]-=-. This phenomenon is very common, however, as the following result shows.sTheorem 5. Let X = S n−1 , n ≥ 3. Let µ be the Lebesgue measure on S n−1 . Let f : [−1, 1] → R be a continuous function, givin... |

37 | Model selection for regularized leastsquares algorithm in learning theory
- Vito, Caponnetto, et al.
- 2005
(Show Context)
Citation Context ...wever, goes far beyond the feature maps that it induces: the eigenvalues and eigenfunctions associated with K play a central role in obtaining error estimates in learning theory, see for example [8], =-=[4]-=-. For this reason, the determination of the spectrum of K, which is highly nontrivial in general, is crucially important in its own right. Theorems 2 and 3 in Section 2 give the complete spectrum of t... |

16 | An explicit description of the reproducing kernel Hilbert spaces of Gaussian RBF kernels
- Steinwart, Hush, et al.
(Show Context)
Citation Context ...e maps for the inverse multiquadric, exponential, ||x−t||2 − or B-spline kernels. The identity e σ2 = ( 4 πσ2 ) n � 2 Rn 2||x−u||2 − e σ2 2||t−u||2 − e σ2 du can also be verified directly, as done in =-=[10]-=-, where implications of the Gaussian feature map Φconv(x) above are also discussed. 4.3 Equivalence of Feature Maps It is known ([7], page 39) that, given a set X and a pointwise-defined, symmetric, p... |

15 | Regularization with dot-product kernels
- Smola, Ovari, et al.
- 2000
(Show Context)
Citation Context ...nd Gaussian kernels. We need the concept of spherical harmonics, a modern and authoritative account of which is [6]. Some of the material below was first reported in the kernel learning literature in =-=[9]-=-, where the eigenvalues for the polynomial kernels with n = 3, were computed. In this section, we will carry out computations for a general n ∈ N, n ≥ 2. � 2 ∂ Definition 1 (Spherical Harmonics). Let ... |