In many applications - such as compression, de-noising and source separation - a good and efficient signal representation is characterized by sparsity. This means that many coefficients are close to zero, while only few ones have a non-negligible amplitude. On the other hand, real-world signals - such as audio or natural images - clearly present peculiar structures. In this paper we introduce a global optimization framework that aims at respecting the sparsity criterion while decomposing a signal over an overcomplete, multi-component dictionary. We adopt a probabilistic analysis which can lead to consider the signal internal structure. As an example that fits this framework, we propose the Weighted Basis Pursuit algorithm, based on the solution of a convex, non-quadratic problem. Results show that this method can provide sparse signal representations and sparse m-terms approximations. Moreover, Weighted Basis Pursuit provides a faster convergence compared to Basis Pursuit.

We present a solution for constructing overcomplete sparse subband decompositions. This is a generalization of the perturbed basis pursuit problem [2] specifically applied to an overcomplete subband representation. Our formulation is based upon the iterative re-weighted least squares algorithm and can be given a probabilistic interpretation. Although the convergence properties of this algorithm are known to be slow, we observe experimentally that only a few iterates are sufficient to generate a reasonably sparse approximation. Furthermore using subband bases provides us with an algorithm whose complexity grows linearly in time.

Abstract—Sparse representations has become an important topic in recent years. It consists in representing, say, a signal (vector) as a linear combination of as few as possible components (vectors) from a redundant basis (of the vector space). This is usually performed, either iteratively (adding a component at a time), or globally (selecting simultaneously all the needed components). We consider a specific algorithm, that we obtain as a fixed point algorithm, but that can also be seen as an iteratively reweighted least-squares algorithm. We analyze it thoroughly and show that it converges to the global optimum. We detail the proof in the real case and indicate how to extend it to the complex case. We illustrate the result with some easily reproducible toy simulations, that further illustrate the potential tracking properties of the proposed algorithm. Index Terms—Convergence of numerical methods, fixed-point algorithms, iterative methods, minimization methods, spectral analysis. I.

Abstract—Sparse signal representations and approximations from overcomplete dictionaries have become an invaluable tool recently. In this paper, we develop a new, heuristic, graph-structured, sparse signal representation algorithm for overcomplete dictionaries that can be decomposed into subdictionaries and whose dictionary elements can be arranged in a hierarchy. Around this algorithm, we construct a methodology for advanced image formation in wide-angle synthetic aperture radar (SAR), defining an approach for joint anisotropy characterization and image formation. Additionally, we develop a coordinate descent method for jointly optimizing a parameterized dictionary and recovering a sparse representation using that dictionary. The motivation is to characterize a phenomenon in wide-angle SAR that has not been given much attention before: migratory scattering centers, i.e., scatterers whose apparent spatial location depends on aspect angle. Finally, we address the topic of recovering solutions that are sparse in more than one objective domain by introducing a suitable sparsifying cost function. We encode geometric objectives into SAR image formation through sparsity in two domains, including the normal parameter space of the Hough transform. Index Terms—Hough transforms, inverse problems, optimization methods, overcomplete dictionaries, sparse signal representations, synthetic aperture radar, tree searching. I.

This paper studies the effect of discretizing the parametrization of a dictionary used for Matching Pursuit decompositions of signals. Our approach relies on viewing the continuously parametrized dictionary as an embedded manifold in the signal space on which the tools of differential (Riemannian) geometry can be applied. The main contribution of this paper is twofold. First, we prove that if a discrete dictionary reaches a minimal density criterion, then the corresponding discrete MP (dMP) is equivalent in terms of convergence to a weakened hypothetical continuous MP. Interestingly, the corresponding weakness factor depends on a density measure of the discrete dictionary. Second, we show that the insertion of a simple geometric gradient ascent optimization on the atom dMP selection maintains the previous comparison but with a weakness factor at least two times closer to unity than without optimization. Finally, we present numerical experiments confirming our theoretical predictions for decomposition of signals and images on regular discretizations of dictionary parametrizations.

In this paper we describe recent advances in harmonic models for musical signal analysis. In particular we consider cases where inharmonic musical instruments are present. A particular model corresponding to a plucked string instrument is adopted and its parameters are automatically estimated from the data. In addition we describe new Bayesian prior distributions for musical pitch, which allow automatic adjustment to the pitch of the instruments, adaptive modelling of harmonic decay across frequency and automatic selection of the basis function window length. The methods are implemented using sophisticated MCMC algorithms and results demonstrated on a small set of examples, including harpsichord and guitar. The methods are readily adapted to polyphonic settings for pitch transcription and can be expected to lead to improvements in that setting. 1.

Abstract. We consider the source separation problem for single-channel music signals. After a brief review of existing methods, we focus on decomposing a mixture into components made of harmonic sinusoidal partials. We address this problem in the Bayesian framework by building a probabilistic model of the mixture combining generic priors for harmonicity, spectral envelope, note duration and continuity. Experiments suggest that the derived blind decomposition method leads to better separation results than nonnegative matrix factorization for certain mixtures. 1

An important component of traffic analysis and network monitoring is the ability to correlate events across multiple data streams, from different sources and from different time periods. Storing such a large amount of data for visualizing traffic trends and for building prediction models of “normal ” network traffic represents a great challenge because the data sets are enormous. In this paper we present the application and analysis of signal processing techniques for effective practical compression of network traffic data. We propose to use a sparse approximation of the network traffic data over a rich collection of natural building blocks, with several natural dictionaries drawn from the networking community’s experience with traffic data. We observe that with such natural dictionaries, high fidelity compression of the original traffic data can be achieved such that even with a compression ratio of around 1:6, the compression error, in terms of the energy of the original signal lost, is less than 1%. We also observe that the sparse representations are stable over time, and that the stable components correspond to well-defined periodicities in network traffic. 1

Hybrid video coding combines together two stages: first motion estimation and compensation predict each frame from the neighboring frames, then the prediction error is coded, reducing the correlation in the spatial domain. In this work we focus on the latter stage, presenting a scheme that profits of some of the features introduced by the standard H.264/AVC for motion estimation and replaces the transform in the spatial domain. The prediction error is so coded using the Matching Pursuit algorithm which decomposes the signal over an appositely designed bi-dimensional, anisotropic, redundant dictionary. Comparisons are made among the proposed technique, H.264 and a DCT-based coding scheme. Moreover, we introduce fast techniques for atom selection, which exploit the spatial localization of the atoms. An adaptive coding scheme aimed at optimizing the resource allocation is also presented, together with a rate-distortion study for the Matching Pursuit algorithm. Results show that the proposed scheme outperforms the standard DCT, especially at very low bit-rates.

This report studies the effect of introducing a priori knowledge to recover sparse representations when overcomplete dictionaries are used. We focus mainly on Greedy algorithms and Basis Pursuit as for our algorithmic basement, while a priori is incorporated by suitably weighting the elements of the dictionary. A unique sufficient condition is provided under which Orthogonal Matching Pursuit, Matching Pursuit and Basis Pursuit are able to recover the optimally sparse representation of a signal when a priori information is available. Theoretical results show how the use of “reliable ” a priori information can improve the performances of these algorithms. In particular, we prove that sufficient conditions to guarantee the retrieval of the sparsest solution can be established for dictionaries unable to satisfy the results of Gribonval and Vandergheynst [1] and Tropp [2]. As one might expect, our results reduce to the classical case of [1] and