Results 1  10
of
18
BLENDENPIK: SUPERCHARGING LAPACK'S LEASTSQUARES SOLVER
"... Several innovative randomsampling and randommixing techniques for solving problems in linear algebra have been proposed in the last decade, but they have not yet made a significant impact on numerical linear algebra. We show that by using an high quality implementation of one of these techniques w ..."
Abstract

Cited by 41 (4 self)
 Add to MetaCart
(Show Context)
Several innovative randomsampling and randommixing techniques for solving problems in linear algebra have been proposed in the last decade, but they have not yet made a significant impact on numerical linear algebra. We show that by using an high quality implementation of one of these techniques we obtain a solver that performs extremely well in the traditional yardsticks of numerical linear algebra: it is significantly faster than highperformance implementations of existing stateoftheart algorithms, and it is numerically backward stable. More speci cally, we describe a leastsquare solver for dense highly overdetermined systems that achieves residuals similar to those of direct QR factorization based solvers (lapack), outperforms lapack by large factors, and scales significantly better than any QRbased solver.
Improved matrix algorithms via the subsampled randomized Hadamard transform
 SIAM J. Matrix Analysis Applications
"... Abstract. Several recent randomized linear algebra algorithms rely upon fast dimension reduction methods. A popular choice is the subsampled randomized Hadamard transform (SRHT). In this article, we address the efficacy, in the Frobenius and spectral norms, of an SRHTbased lowrank matrix approxim ..."
Abstract

Cited by 17 (3 self)
 Add to MetaCart
(Show Context)
Abstract. Several recent randomized linear algebra algorithms rely upon fast dimension reduction methods. A popular choice is the subsampled randomized Hadamard transform (SRHT). In this article, we address the efficacy, in the Frobenius and spectral norms, of an SRHTbased lowrank matrix approximation technique introduced by Woolfe, Liberty, Rohklin, and Tygert. We establish a slightly better Frobenius norm error bound than is currently available, and a much sharper spectral norm error bound (in the presence of reasonable decay of the singular values). Along the way, we produce several results on matrix operations with SRHTs (such as approximate matrix multiplication) that may be of independent interest. Our approach builds upon Tropp’s in “Improved Analysis of the
Accelerated lowrank visual recovery by random projection
 in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR
"... Exact recovery from contaminated visual data plays an important role in various tasks. By assuming the observed data matrix as the addition of a lowrank matrix and a sparse matrix, theoretic guarantee exists under mild conditions for exact data recovery. Practically matrix nuclear norm is adopted ..."
Abstract

Cited by 15 (0 self)
 Add to MetaCart
(Show Context)
Exact recovery from contaminated visual data plays an important role in various tasks. By assuming the observed data matrix as the addition of a lowrank matrix and a sparse matrix, theoretic guarantee exists under mild conditions for exact data recovery. Practically matrix nuclear norm is adopted as a convex surrogate of the nonconvex matrix rank function to encourage lowrank property and serves as the major component of recentlyproposed Robust Principal Component Analysis (RPCA). Recent endeavors have focused on enhancing the scalability of RPCA to largescale datasets, especially mitigating the computational burden of frequent largescale Singular Value Decomposition (SVD) inherent with the nuclear norm optimization. In our proposed scheme, the nuclear norm of an auxiliary matrix is minimized instead, which is related to the original lowrank matrix by random projection. By design, the modified optimization entails SVD on matrices of much smaller scale, as compared to the original optimization problem. Theoretic analysis well justifies the proposed scheme, along with greatly reduced optimization complexity. Both qualitative and quantitative studies are provided on various computer vision benchmarks to validate its effectiveness, including facial shadow removal, surveillance background modeling and largescale image tag transduction. It is also highlighted that the proposed solution can serve as a general principal to accelerate many other nuclear norm oriented problems in numerous tasks. 1.
Efficient Nonnegative Matrix Factorization with Random Projections
"... The recent years have witnessed a surge of interests in Nonnegative Matrix Factorization (NMF) in data mining and machine learning fields. Despite its elegant theory and empirical success, one of the limitations of NMF based algorithms is that it needs to store the whole data matrix in the entire pr ..."
Abstract

Cited by 8 (5 self)
 Add to MetaCart
(Show Context)
The recent years have witnessed a surge of interests in Nonnegative Matrix Factorization (NMF) in data mining and machine learning fields. Despite its elegant theory and empirical success, one of the limitations of NMF based algorithms is that it needs to store the whole data matrix in the entire process, which requires expensive storage and computation costs when the data set is large and highdimensional. In this paper, we propose to apply the random projection techniques to accelerate the NMF process. Both theoretical analysis and experimental validations will be presented to demonstrate the effectiveness of the proposed strategy. 1
The effect of coherence on sampling from matrices with orthonormal columns, and preconditioned least squares problems. arXiv preprint arXiv:1203.4809
, 2012
"... ar ..."
(Show Context)
Nearoptimal coresets for leastsquares regression
 IEEE Transactions on Information Theory
, 2013
"... Abstract—We study the (constrained) leastsquares regression as well asmultiple response leastsquares regression and ask the question of whether a subset of the data, a coreset, suffices to compute a good approximate solution to the regression. We give deterministic, loworder polynomialtime alg ..."
Abstract

Cited by 5 (1 self)
 Add to MetaCart
(Show Context)
Abstract—We study the (constrained) leastsquares regression as well asmultiple response leastsquares regression and ask the question of whether a subset of the data, a coreset, suffices to compute a good approximate solution to the regression. We give deterministic, loworder polynomialtime algorithms to construct such coresets with approximation guarantees, together with lower bounds indicating that there is not much room for improvement upon our results. Index Terms—Least mean square algorithms, machine learning algorithms, regression analysis. I.
Efficient Dimensionality Reduction for Canonical Correlation Analysis
"... We present a fast algorithm for approximate Canonical Correlation Analysis (CCA). Given a pair of tallandthin matrices, the proposed algorithm first employs a randomized dimensionality reduction transform to reduce the size of the input matrices, and then applies any standard CCA algorithm to the ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
(Show Context)
We present a fast algorithm for approximate Canonical Correlation Analysis (CCA). Given a pair of tallandthin matrices, the proposed algorithm first employs a randomized dimensionality reduction transform to reduce the size of the input matrices, and then applies any standard CCA algorithm to the new pair of matrices. The algorithm computes an approximate CCA to the original pair of matrices with provable guarantees, while requiring asymptotically less operations than the stateoftheart exact algorithms. 1.
Iterative Hessian Sketch: Fast and Accurate Solution Approximation for Constrained LeastSquares
, 2016
"... Abstract We study randomized sketching methods for approximately solving leastsquares problem with a general convex constraint. The quality of a leastsquares approximation can be assessed in different ways: either in terms of the value of the quadratic objective function (cost approximation), or ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
Abstract We study randomized sketching methods for approximately solving leastsquares problem with a general convex constraint. The quality of a leastsquares approximation can be assessed in different ways: either in terms of the value of the quadratic objective function (cost approximation), or in terms of some distance measure between the approximate minimizer and the true minimizer (solution approximation). Focusing on the latter criterion, our first main result provides a general lower bound on any randomized method that sketches both the data matrix and vector in a leastsquares problem; as a surprising consequence, the most widely used leastsquares sketch is suboptimal for solution approximation. We then present a new method known as the iterative Hessian sketch, and show that it can be used to obtain approximations to the original leastsquares problem using a projection dimension proportional to the statistical complexity of the leastsquares minimizer, and a logarithmic number of iterations. We illustrate our general theory with simulations for both unconstrained and constrained versions of leastsquares, including 1 regularization and nuclear norm constraints. We also numerically demonstrate the practicality of our approach in a real face expression classification experiment.