Citation Context

...γ ) [ ⋆ ⋆ ξ(B ) − ξ(B ) < 1 − 2ξ(B⋆ )µ(A⋆ ] + γ ) = γ. Here, we used the fact that ξ(B ⋆ ) 1−4ξ(B⋆ )µ(A⋆) < γ in the second inequality. □ Proof of Proposition 3. Based on the Perron-Frobenius theorem =-=[18]-=-, one can conclude that ‖P ‖ ≥ ‖Q‖ if Pi,j ≥ |Qi,j|, ∀ i, j. Thus, we need only consider the matrix that has 1 in every location in the support set Ω(A) and 0 everywhere else. Based on the definition ...

Citation Context

...n-space orthogonal to the column-space of B ⋆ . We have that P T (B ⋆ ) ⊥(M) = (In×n − PU )M(In×n − PV ), (4.2) where In×n is the n × n identity matrix. Following standard notation in convex analysis =-=[26]-=-, we denote the subdifferential of a convex function f at a point ˆx in its domain by ∂f(ˆx). The subdifferential ∂f(ˆx) consists of all y such that f(x) ≥ f(ˆx) + 〈y, x − ˆx〉, ∀x. From the optimality...

Citation Context

...ecovering sparse solutions [9]. Incoherence is also a concept that is used in recent work under the title of compressed sensing, which aims to recover “low-dimensional” objects such as sparse vectors =-=[3, 12]-=- and low-rank matrices [24, 4] given incomplete observations. Our work is closer in spirit to that in [10], and can be viewed as a method to recover the “simplest explanation” of a matrix given an “ov...

Citation Context

...ecovering sparse solutions [9]. Incoherence is also a concept that is used in recent work under the title of compressed sensing, which aims to recover “low-dimensional” objects such as sparse vectors =-=[3, 12]-=- and low-rank matrices [24, 4] given incomplete observations. Our work is closer in spirit to that in [10], and can be viewed as a method to recover the “simplest explanation” of a matrix given an “ov...

Citation Context

...of A⋆ . We have that degmax(A ⋆ ) ≤ m n log(n), with high probability. The proof of this lemma follows from a standard balls and bins argument, and can be found in several references (see for example =-=[2]-=-). Next we consider low-rank matrices in which the singular vectors are chosen uniformly at random from the set of all partial isometries. Such a model was considered in recent work on the matrix comp...

Citation Context

...he sparse and low-rank matrices having different interpretations depending on the application. In a statistical model selection setting, the sparse matrix can correspond to a Gaussian graphical model =-=[19]-=- and the low-rank matrix can summarize the effect of latent, unobserved variables. Decomposing a given model into these simpler components is useful for developing efficient estimation and inference a...

Citation Context

... C. (1.3) Here γ is a parameter that provides a trade-off between the low-rank and sparse components. This optimization problem is convex, and can in fact be rewritten as a semidefinite program (SDP) =-=[31]-=- (see Appendix A). We prove that ( Â, ˆ B) = (A⋆, B⋆ ) is the unique optimum of (1.3) for a range of γ if µ(A⋆)ξ(B ⋆ ) < 1 6 (see Theorem 2 in Section 4.2). Thus, the conditions for4 V. Chandrasekara...

Citation Context

...rtially coherent decomposition in optical systems We outline an optics application that is described in greater detail in [13]. Optical imaging systems are commonly modeled using the Hopkins integral =-=[16]-=-, which gives the output intensity at a point as a function of the input transmission via a quadratic form. In many applications the operator in this quadratic form can be well-approximated by a (fini...

Citation Context

...t was used to recover low-rank positive semidefinite matrices [22]. Indeed, several papers demonstrate that the nuclear norm heuristic recovers low-rank matrices in various rank minimization problems =-=[24, 4]-=-. Based on these results, we propose the following optimization formulation to recover A ⋆ and B ⋆ given C = A ⋆ + B ⋆ : ( Â, ˆ B) = arg min A,B γ‖A‖1 + ‖B‖∗ s.t. A + B = C. (1.3) Here γ is a paramete...

Citation Context

...r such matrices. 1.2. Previous work using incoherence. The concept of incoherence was studied in the context of recovering sparse representations of vectors from a so-called “overcomplete dictionary” =-=[10]-=-. More concretely consider a situation in which one is given a vector formed by a sparse linear combination of a few elements from a combined time-frequency dictionary, i.e., a vector formed by adding...

Citation Context

...nd sinusoids that compose the vector from the infinitely many possible solutions. Based on a notion of time-frequency incoherence, the ℓ1 heuristic was shown to succeed in recovering sparse solutions =-=[9]-=-. Incoherence is also a concept that is used in recent work under the title of compressed sensing, which aims to recover “low-dimensional” objects such as sparse vectors [3, 12] and low-rank matrices ...

Citation Context

...d as an effective surrogate for the number of nonzero entries of a vector, and a number of results provide conditions under which this heuristic recovers sparse solutions to illposed inverse problems =-=[11]-=-. More recently, the nuclear norm has been shown to be an effective surrogate for the rank of a matrix [14]. This relaxation is a generalization of the previously studied trace-heuristic that was used...

Citation Context

...t was used to recover low-rank positive semidefinite matrices [22]. Indeed, several papers demonstrate that the nuclear norm heuristic recovers low-rank matrices in various rank minimization problems =-=[24, 4]-=-. Based on these results, we propose the following optimization formulation to recover A ⋆ and B ⋆ given C = A ⋆ + B ⋆ : ( Â, ˆ B) = arg min A,B γ‖A‖1 + ‖B‖∗ s.t. A + B = C. (1.3) Here γ is a paramete...

Citation Context

...nfirm the theoretical predictions in this paper with some simple experimental results. We also present a heuristic to choose the trade-off parameter γ. All our simulations were performed using YALMIP =-=[20]-=- and the SDPT3 software [30] for solving SDPs. Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.586 CHANDRASEKARAN, SANGHAVI, PARRILO, AND WILLSKY Downloaded 08/14/12 to 1...

Citation Context

...tions in this paper with some simple experimental results. We also present a heuristic to choose the trade-off parameter γ. All our simulations were performed using YALMIP [20] and the SDPT3 software =-=[30]-=- for solving SDPs. Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.586 CHANDRASEKARAN, SANGHAVI, PARRILO, AND WILLSKY Downloaded 08/14/12 to 128.31.5.23. Redistribution s...

Citation Context

...conditions under which this heuristic recovers sparse solutions to illposed inverse problems [11]. More recently, the nuclear norm has been shown to be an effective surrogate for the rank of a matrix =-=[14]-=-. This relaxation is a generalization of the previously studied trace-heuristic that was used to recover low-rank positive semidefinite matrices [22]. Indeed, several papers demonstrate that the nucle...

Citation Context

...rn of a matrix and its row/column spaces. This condition is based on quantities involving the tangent spaces to the algebraic variety of sparse matrices and the algebraic variety of low-rank matrices =-=[17]-=-. Another point of ambiguity in the problem statement is that one could subtract a nonzero entry from A and add it to B ; the sparsity level of A is strictly improved, while the rank of B is increased...

Citation Context

... also be posed in the system identification setting. Linear time-invariant (LTI) systems can be represented by Hankel matrices, where the matrix represents the input-output relationship of the system =-=[28]-=-. Thus, a sparse Hankel matrix corresponds to an LTI system with a sparse impulse response. A low-rank Hankel matrix corresponds to a system with small model order, and provides a minimal realization ...

Citation Context

... (1.3) yields exact recovery with high probability even when the size of the support of A⋆ is super-linear in n. During final preparation of this manuscript we learned of related contemporaneous work =-=[30]-=- that specifically studies the problem of decomposing random sparse and low-rank matrices. In addition to the assumptions of our random sparsity and random orthogonal models, [30] also requires that t...

Citation Context

...d variables. Decomposing a given model into these simpler components is useful for developing efficient estimation and inference algorithms. In computational complexity, the notion of matrix rigidity =-=[30]-=- captures the smallest number of entries of a matrix that must be changed in order to reduce the rank of the matrix below a specified level (the changes can be of arbitrary magnitude). Bounds on the r...

Citation Context

...rse Hankel matrix corresponds to an LTI system with a sparse impulse response. A low-rank Hankel matrix corresponds to a system with small model order, and provides a minimal realization for a system =-=[15]-=-. Given an LTI system H as follows H = Hs + Hlr,6 V. Chandrasekaran, S. Sanghavi, P. A. Parrilo, and A. S. Willsky where Hs is sparse and Hlr is low-rank, obtaining a simple description of H requires...

Citation Context

...‖1 if and only if P Ω(A ⋆ )(Q) = γ sign(A ⋆ ), ‖P Ω(A ⋆ ) c(Q)‖∞ ≤ γ. (4.4) Here sign(A⋆ i,j ) equals +1 if A⋆i,j > 0, −1 if A⋆i,j < 0, and 0 if A⋆i,j = 0. We also have that Q ∈ ∂‖B⋆‖∗ if and only if =-=[32]-=- P T (B ⋆ )(Q) = UV ′ , ‖P T (B ⋆ ) ⊥(Q)‖ ≤ 1. (4.5) Note that these are necessary and sufficient conditions for (A ⋆ , B ⋆ ) to be an optimum of (1.3). The following proposition provides sufficient c...

Citation Context

...te µ(A) as follows: ∑ µ(A) = max xiyj. (B.15) ‖x‖2=1,‖y‖2=1 (i,j)∈Ω(A) ⎠22 V. Chandrasekaran, S. Sanghavi, P. A. Parrilo, and A. S. Willsky Upper bound. For any matrix M, we have from the results in =-=[27]-=- that ‖M‖ 2 ≤ max ricj, (B.16) i,j where ri = ∑ k |Mi,k| denotes the absolute row-sum of row i and cj = ∑ k |Mk,j| denotes the absolute column-sum of column j. Let M Ω(A) be a matrix defined as follow...

Citation Context

...tially coherent decomposition in optical systems. We outline an optics application that is described in greater detail in [13]. Optical imaging systems are commonly modeled using the Hopkins integral =-=[16]-=-, which gives the output intensity at a point as a function of the input transmission via a quadratic form. In many applications the operator in this quadratic form can be well-approximated by a (fini...

Citation Context

...be an effective surrogate for the rank of a matrix [14]. This relaxation is a generalization of the previously studied trace-heuristic that was used to recover low-rank positive semidefinite matrices =-=[22]-=-. Indeed, several papers demonstrate that the nuclear norm heuristic recovers low-rank matrices in various rank minimization problems [24, 4]. Based on these results, we propose the following optimiza...

Citation Context

...tions in this paper with some simple experimental results. We also present a heuristic to choose the trade-off parameter γ. All our simulations were performed using YALMIP [33] and the SDPT3 software =-=[29]-=- for solving SDPs. In the first experiment we generate random 25 × 25 matrices according to the random sparsity and random orthogonal models described in Section 4.4. To generate a random rank-k matri...

Citation Context

...dentification [6]. Submitted on June 11, 2009; revised on October 4, 2010. 12 V. Chandrasekaran, S. Sanghavi, P. A. Parrilo, and A. S. Willsky a matrix have several implications in complexity theory =-=[20]-=-. Similarly, in a system identification setting the low-rank matrix represents a system with a small model order while the sparse matrix represents a system with a sparse impulse response. Decomposing...

Citation Context

...rn of a matrix and its row/column spaces. This condition is based on quantities involving the tangent spaces to the algebraic variety of sparse matrices and the algebraic variety of low-rank matrices =-=[17]-=-. Another point of ambiguity in the problem statement is that one could subtract a nonzero entry from A ⋆ and add it to B ⋆ ; the sparsity level of A ⋆ is strictly improved while the rank of B ⋆ is in...

Citation Context

...hat certain tangent spaces have a transverse intersection. The implications of our results for the matrix rigidity problem are also demonstrated. We would like to mention here a related piece of work =-=[5]-=- that appeared subsequent to the submission of our paper. In [5] the authors analyze the convex program (1.3) for the sparse-plus-low-rank decomposition problem Copyright © by SIAM. Unauthorized repro...

Citation Context

...herent imaging systems, and the corresponding system matrices have small rank. For systems that are not perfectly coherent various methods have been proposed to find an optimal coherent decomposition =-=[23]-=-, and these essentially identify the best approximation of the system matrix by a matrix of lower rank. At the other end are incoherent optical systems that allow some high frequencies, and are charac...

Citation Context

...ion. The implications of ourRank-Sparsity Incoherence for Matrix Decomposition 17 results for the matrix rigidity problem are also demonstrated. We would like to mention here a related piece of work =-=[5]-=- that appeared subsequent to the submission of our paper. In [5] the authors analyze the convex program (1.3) for the sparse-pluslow-rank decomposition problem, and provide results for exact recovery ...

Citation Context

...nstraint that the sparse and low-rank matrices have Hankel structure. 2.4. Partially coherent decomposition in optical systems. We outline an optics application that is described in greater detail in =-=[13]-=-. Optical imaging systems are commonly modeled using the Hopkins integral [16], which gives the output intensity at a point as a function of the input transmission via a quadratic form. In many applic...

Citation Context

...ity has a number of implications in complexity theory [20], such as the trade-offs between size and depth in arithmetic circuits. However, computing the rigidity of a matrix is intractable in general =-=[21, 8]-=-. For any M ∈ R n×n one can check that RM (k) ≤ (n − k) 2 (this follows directly from a Schur complement argument). Generically every M ∈ R n×n is very rigid, i.e., RM (k) = (n − k) 2 [30], although s...

Citation Context

... http://www.siam.org/journals/ojsa.php spanðUÞ. We show in Appendix B that matrices with incoherent row/column spaces have small ξ; the proof technique for the lower bound here was suggested by Recht =-=[26]-=-. PROPOSITION 4. Let B ∈ R n×n be any matrix with incðBÞ defined as in (4.7) and ξðBÞ defined as in (1.1). We have that incðBÞ ≤ ξðBÞ ≤ 2 incðBÞ: If B ∈ Rn×n is a full-rank matrix or a matrix such as ...

Citation Context

...ity has a number of implications in complexity theory [20], such as the trade-offs between size and depth in arithmetic circuits. However, computing the rigidity of a matrix is intractable in general =-=[21, 8]-=-. For any M ∈ R n×n one can check that RM (k) ≤ (n − k) 2 (this follows directly from a Schur complement argument). Generically every M ∈ R n×n is very rigid, i.e., RM (k) = (n − k) 2 [30], although s...

Citation Context

...als the graphical structure in the observed variables as well as the effect due to (and the number of) the unobserved latent variables. We discuss this application in more detail in a separate report =-=[7]-=-. 2.2. Matrix rigidity. The rigidity of a matrix M, denoted by RM (k), is the smallest number of entries that need to be changed in order to reduce the rank of M below k. Obtaining bounds on rigidity ...

Citation Context

...s a number of implications in complexity theory [21], such as the trade-offs between size and depth in arithmetic circuits. However, computing the rigidity of a matrix is intractable in general [22], =-=[8]-=-. For any M ∈ Rn×n one can check that RM ðkÞ ≤ ðn − kÞ2 (this follows directly from a Schur complement argument). Generically every M ∈ Rn×n is very rigid, i.e., RM ðkÞ ðn− kÞ2 [31], although special...

Citation Context

...ity has a number of implications in complexity theory [21], such as the trade-offs between size and depth in arithmetic circuits. However, computing the rigidity of a matrix is intractable in general =-=[22]-=-, [8]. For any M ∈ Rn×n one can check that RM ðkÞ ≤ ðn − kÞ2 (this follows directly from a Schur complement argument). Generically every M ∈ Rn×n is very rigid, i.e., RM ðkÞ ðn− kÞ2 [31], although sp...