Results 1 
5 of
5
Improved bounds on sample size for implicit matrix trace estimators
, 2013
"... This article is concerned with MonteCarlo methods for the estimation of the trace of an implicitly given matrix A whose information is only available through matrixvector products. Such a method approximates the trace by an average of N expressions of the form wt(Aw), with random vectors w drawn f ..."
Abstract

Cited by 9 (3 self)
 Add to MetaCart
(Show Context)
This article is concerned with MonteCarlo methods for the estimation of the trace of an implicitly given matrix A whose information is only available through matrixvector products. Such a method approximates the trace by an average of N expressions of the form wt(Aw), with random vectors w drawn from an appropriate distribution. We prove, discuss and experiment with bounds on the number of realizations N required in order to guarantee a probabilistic bound on the relative error of the trace estimation upon employing Rademacher (Hutchinson), Gaussian and uniform unit vector (with and without replacement) probability distributions. In total, one necessary bound and six sufficient bounds are proved, improving upon and extending similar estimates obtained in the seminal work of Avron and Toledo (2011) in several dimensions. We first improve their bound on N for the Hutchinson method, dropping a term that relates to rank(A) and making the bound comparable with that for the Gaussian estimator. We further prove new sufficient bounds for the Hutchinson, Gaussian and the unit vector estimators, as well as a necessary bound for the Gaussian estimator, which depend more specifically on properties of the matrix A. As such they may suggest for what type of matrices one distribution or another provides a particularly effective or relatively ineffective stochastic estimation method.
The lost honour of ℓ2based regularization
, 2012
"... In the past two decades, regularization methods based on the ℓ1 norm, including sparse wavelet representations and total variation, have become immensely popular. So much so, that we were led to consider the question whether ℓ1based techniques ought to altogether replace the simpler, faster and bet ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
In the past two decades, regularization methods based on the ℓ1 norm, including sparse wavelet representations and total variation, have become immensely popular. So much so, that we were led to consider the question whether ℓ1based techniques ought to altogether replace the simpler, faster and better known ℓ2based alternatives as the default approach to regularization techniques. The occasionally tremendous advances of ℓ1based techniques are not in doubt. However, such techniques also have their limitations. This article explores advantages and disadvantages compared to ℓ2based techniques using several practical case studies. Taking into account the considerable added hardship in calculating solutions of the resulting computational problems, ℓ1based techniques must offer substantial advantages to be worthwhile. In this light our results suggest that in many applications, though not all, ℓ2based recovery may still be preferred. 1
Data completion and stochastic algorithms for PDE inversion problems with many measurements
, 2013
"... Inverse problems involving systems of partial differential equations (PDEs) with many measurements or experiments can be very expensive to solve numerically. In a recent paper we examined dimensionality reduction methods, both stochastic and deterministic, to reduce this computational burden, assumi ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
Inverse problems involving systems of partial differential equations (PDEs) with many measurements or experiments can be very expensive to solve numerically. In a recent paper we examined dimensionality reduction methods, both stochastic and deterministic, to reduce this computational burden, assuming that all experiments share the same set of receivers. In the present article we consider the more general and practically important case where receivers are not shared across experiments. We propose a data completion approach to alleviate this problem. This is done by means of an approximation using a gradient or Laplacian regularization, extending existing data for each experiment to the union of all receiver locations. Results using the method of simultaneous sources with the completed data are then compared to those obtained by a more general but slower random subset method which requires no modifications. 1
ASSESSING STOCHASTIC ALGORITHMS FOR LARGE SCALE NONLINEAR LEAST SQUARES PROBLEMS USING EXTREMAL PROBABILITIES OF LINEAR COMBINATIONS OF GAMMA RANDOM VARIABLES
"... Abstract. This article considers stochastic algorithms for efficiently solving a class of large scale nonlinear least squares (NLS) problems which frequently arise in applications. We propose eight variants of a practical randomized algorithm where the uncertainties in the major stochastic steps ar ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
Abstract. This article considers stochastic algorithms for efficiently solving a class of large scale nonlinear least squares (NLS) problems which frequently arise in applications. We propose eight variants of a practical randomized algorithm where the uncertainties in the major stochastic steps are quantified. Such stochastic steps involve approximating the NLS objective function using MonteCarlo methods, and this is equivalent to the estimation of the trace of corresponding symmetric positive semidefinite (SPSD) matrices. For the latter, we prove tight necessary and sufficient conditions on the sample size (which translates to cost) to satisfy the prescribed probabilistic accuracy. We show that these conditions are practically computable and yield small sample sizes. They are then incorporated in our stochastic algorithm to quantify the uncertainty in each randomized step. The bounds we use are applications of more general results regarding extremal tail probabilities of linear combinations of gamma distributed random variables. We derive and prove new results concerning the maximal and minimal tail probabilities of such linear combinations, which can be considered independently of the rest of this paper.
ALGORITHMS THAT SATISFY A STOPPING CRITERION, PROBABLY
"... Iterative numerical algorithms are typically equipped with a stopping criterion, where the iteration process is terminated when some error or misfit measure is deemed to be below a given tolerance. This is a useful setting for comparing algorithm performance, among other purposes. However, in practi ..."
Abstract
 Add to MetaCart
(Show Context)
Iterative numerical algorithms are typically equipped with a stopping criterion, where the iteration process is terminated when some error or misfit measure is deemed to be below a given tolerance. This is a useful setting for comparing algorithm performance, among other purposes. However, in practical applications a precise value for such a tolerance is rarely known; rather, only some possibly vague idea of the desired quality of the numerical approximation is at hand. This leads us to think of ways to relax the notion of exactly satisfying a tolerance value, in a hopefully profitable way. We give wellknown examples where a deterministic relaxation of this notion is applied. Another possibility which we concentrate on is a probabilistic relaxation of the given tolerance. This allows, for instance, derivation of proven bounds on the sample size of certain Monte Carlo methods. We describe this in the context of particular applications.