## Information cost tradeoffs for augmented index and streaming language recognition (2010)

### Cached

### Download Links

Venue: | In Proc. IEEE Symposium on Foundations of Computer Science |

Citations: | 4 - 1 self |

### BibTeX

@INPROCEEDINGS{Chakrabarti10informationcost,

author = {Amit Chakrabarti and Graham Cormode and Ranganath Kondapally and Andrew Mcgregor},

title = {Information cost tradeoffs for augmented index and streaming language recognition},

booktitle = {In Proc. IEEE Symposium on Foundations of Computer Science},

year = {2010},

pages = {387--396}

}

### OpenURL

### Abstract

This paper makes three main contributions to the theory of communication complexity and stream computation. First, we present new bounds on the information complexity of AUGMENTED-INDEX. In contrast to analogous results for INDEX by Jain, Radhakrishnan and Sen [J. ACM, 2009], we have to overcome the significant technical challenge that protocols for AUGMENTED-INDEX may violate the “rectangle property ” due to the inherent input sharing. Second, we use these bounds to resolve an open problem of Magniez, Mathieu and Nayak [STOC, 2010] that asked about the multi-pass complexity of recognizing Dyck languages. This results in a natural separation between the standard multi-pass model and the multi-pass model that permits reverse passes. Third, we present the first passive memory checkers that verify the interaction transcripts of priority queues, stacks, and double-ended queues. We obtain tight upper and lower bounds for these problems, thereby addressing an important sub-class of the memory checking framework of Blum et al. [Algorithmica, 1994].

### Citations

629 | Communication Complexity
- Kushilevitz, Nisan
- 1997
(Show Context)
Citation Context ...dex problem, which leads to these multipass lower bounds. Background, augmented index, and a new lower bound. The index problem is one of a handful of fundamental problems in communication complexity =-=[20]-=-: Alice has ∗Received by the editors November 30, 2010; accepted for publication (in revised form) August 21, 2012; published electronically January 10, 2013. A preliminary version of this work appear... |

154 | An information statistics approach to data stream and communication complexity
- Bar-Yossef, Jayram, et al.
(Show Context)
Citation Context ...resent information costs. Such an information cost tradeoff opens up interesting possibilities for applications to lower bounds for more complex problems via the direct sum properties of this measure =-=[4, 9]-=-. One such application is the aforementioned dyck(2) lower bound. However, the tradeoff theorem of Jain, Radhakrishnan, and Sen [18] is not strong enough to obtain the required direct sum result. One ... |

111 | Multi-party protocols
- Chandra, Furst, et al.
- 1983
(Show Context)
Citation Context ...cal hurdle in this proof is dealing with the fact that Alice and Bob share some of the input, which breaks the useful “rectangle property.” (This is reminiscent of number-on-theforehead communication =-=[10]-=-, where input sharing makes strong lower bounds rather hard to prove.) The highlight of our proof is the fat transcript lemma (Lemma 2.6), with its careful interplay between a suitably weakened rectan... |

99 | Checking the correctness of memories
- Blum, Evans, et al.
- 1994
(Show Context)
Citation Context ...rst multipass space lower bounds for several languages, including dyck(2), resolving an open question of theirs. We also study the complexity of languages that arise in the context of memory checking =-=[5]-=- and present tight upper and lower bounds for them. Our key technical contributions rely on a new understanding of the information complexity of the augmented index problem, which leads to these multi... |

83 | On data structures and asymmetric communication complexity - Miltersen, Nisan, et al. - 1998 |

72 | Near-optimal lower bounds on the multi-party communication complexity of set disjointness
- Chakrabarti, Khot, et al.
- 2003
(Show Context)
Citation Context ...not learn too much about Bob’s input. For each k ∈ [n], we can define a certain set of excluded strings Zk ⊆ {0, 1} n . Then, we can show that ∀ k ∈ K ∀ x ∈ X \ Zk : M(x) ≥ 1 Pr[K ≥ k | X = x] − 2δ . =-=(5)-=- 2 Finally, we can show that these conditions on k and x fail only with low probability and that K is often large. This will then imply that, for the setting δ = 1/D, we have E[M(X)] ≥ 1 6 − , 2 D (6)... |

62 | Informational complexity and the direct sum problem for simultaneous message complexity
- Chakrabarti, Shi, et al.
- 2001
(Show Context)
Citation Context ...resent information costs. Such an information cost tradeoff opens up interesting possibilities for applications to lower bounds for more complex problems via the direct sum properties of this measure =-=[4, 9]-=-. One such application is the aforementioned dyck(2) lower bound. However, the tradeoff theorem of Jain, Radhakrishnan, and Sen [18] is not strong enough to obtain the required direct sum result. One ... |

59 | Optimal space lower bounds for all frequency moments
- Woodruff
- 2004
(Show Context)
Citation Context ... Feigenbaum et al. [9] on lower bounds for various graph streaming problems including connectivity and diameter approximation, Woodruff’s lower bound for approximating frequent items in a data stream =-=[20]-=-, and cell probe lower bounds of Miltersen et al. [17] and Pǎtras¸cu [18]. Thanks to its fundamental role and great importance, the INDEX problem and its variants — in particular, the AUGMENTED-INDEX ... |

55 | Numerical linear algebra in the streaming model
- Clarkson, Woodruff
- 2009
(Show Context)
Citation Context ...motivated by a specific application. Augmentation has been used in the study of sketching complexity [2] and for proving tight space lower bounds for a wide range of problems in the data stream model =-=[7,8,14]-=-. Information cost was formally introduced by Chakrabarti et al. [6] as a technique for proving direct sum theorems in communication and, in the context of INDEX, was introduced by Jain et al. [12] as... |

37 | The complexity of online memory checking
- Naor, Rothblum
- 2005
(Show Context)
Citation Context ...tion is to memory checking, whose study was initiated by Blum et al. [5] and continued by numerous groups including Ajtai [2], Chu, Kannan, and McGregor [11], Dwork et al. [14], and Naor and Rothblum =-=[23]-=-. The problem, as considered in this paper, is to observe a sequence of N updates and queries to (an implementation of) a data structure and to report whether or not the implementation operated correc... |

33 | Lower bounds for sparse recovery
- Ba, Indyk, et al.
- 2010
(Show Context)
Citation Context ...nd Bob have much more “help,” in two ways. First, we relax index so that Bob additionally gets to see the length-(k − 1) prefix of Alice’s input; the resulting variant has been called augmented-index =-=[13, 19]-=-, and the one-way communication lower bound easily extends to it [3]. Second, in our variant of augmented-index, Bob also gets a check bit c ∈{0, 1} and must verify that xk = c. This second twist clea... |

30 | Lower bounds for one-way probabilistic communication complexity and their application to space complexity
- Ablayev
- 1996
(Show Context)
Citation Context ...(n) communication—when messages go only from Alice to Bob, and is “easy”—solvable using O(log n) communication—without this restriction. The lower bound extends to randomized constant-error protocols =-=[1]-=-. This makes index the canonical hard-for-one-way, easy-for-two-way communication problem. Is there really anything new to say about such a fundamental problem? As it turns out, there is, provided one... |

25 |
Shmuel Safra, and Avi Wigderson. On data structures and asymmetric communication complexity
- Miltersen, Nisan
- 1998
(Show Context)
Citation Context ...ph streaming problems including connectivity and diameter approximation, Woodruff’s lower bound for approximating frequent items in a data stream [20], and cell probe lower bounds of Miltersen et al. =-=[17]-=- and Pǎtras¸cu [18]. Thanks to its fundamental role and great importance, the INDEX problem and its variants — in particular, the AUGMENTED-INDEX problem [8,14] , which we study here — have been subje... |

23 | Graph distances in the data-stream model
- Feigenbaum, Kannan, et al.
(Show Context)
Citation Context ...data structures. For instance, the works of Jayram et al. [13] and Kane et al. [14] on the 1-pass space complexity of approximating frequency moments and Lp norm (for 1 ≤ p ≤ 2), of Feigenbaum et al. =-=[9]-=- on lower bounds for various graph streaming problems including connectivity and diameter approximation, Woodruff’s lower bound for approximating frequent items in a data stream [20], and cell probe l... |

18 | On the Exact Space Complexity of Sketching and Streaming Small Norms
- Kane, Nelson, et al.
- 2010
(Show Context)
Citation Context ...nd Bob have much more “help,” in two ways. First, we relax index so that Bob additionally gets to see the length-(k − 1) prefix of Alice’s input; the resulting variant has been called augmented-index =-=[13, 19]-=-, and the one-way communication lower bound easily extends to it [3]. Second, in our variant of augmented-index, Bob also gets a check bit c ∈{0, 1} and must verify that xk = c. This second twist clea... |

13 | The sketching complexity of pattern matching
- Bar-Yossef, Jayram, et al.
- 2004
(Show Context)
Citation Context ...Bob additionally gets to see the length-(k − 1) prefix of Alice’s input; the resulting variant has been called augmented-index [13, 19], and the one-way communication lower bound easily extends to it =-=[3]-=-. Second, in our variant of augmented-index, Bob also gets a check bit c ∈{0, 1} and must verify that xk = c. This second twist clearly does not matter when considering communication complexity, but f... |

12 | How efficient can memory checking be
- Dwork, Naor, et al.
- 2009
(Show Context)
Citation Context ...re allowed. A second application is to memory checking, whose study was initiated by Blum et al. [5] and continued by numerous groups including Ajtai [2], Chu, Kannan, and McGregor [11], Dwork et al. =-=[14]-=-, and Naor and Rothblum [23]. The problem, as considered in this paper, is to observe a sequence of N updates and queries to (an implementation of) a data structure and to report whether or not the im... |

11 | Unifying the landscape of cell-probe lower bounds
- Pǎtras¸cu
- 2008
(Show Context)
Citation Context ...e analogous to the “large rectangles” seen in elementary communication complexity. This part of the proof is reminiscent of arguments in Pǎtra¸scu’s proof of the lopsided set disjointness lower bound =-=[24]-=-. Next, and more interestingly, we use these fatness properties to derive a contradiction in Lemma 2.6. Throughout the proof and the rest of this section, we tacitly assume that n is large enough. Ass... |

10 |
The Invasiveness of Off-Line Memory Checking
- Ajtai
(Show Context)
Citation Context ...ly harder if only multiple unidirectional passes are allowed. A second application is to memory checking, whose study was initiated by Blum et al. [5] and continued by numerous groups including Ajtai =-=[2]-=-, Chu, Kannan, and McGregor [11], Dwork et al. [14], and Naor and Rothblum [23]. The problem, as considered in this paper, is to observe a sequence of N updates and queries to (an implementation of) a... |

8 | Recognizing well-parenthesized expressions in the streaming model
- Magniez, Mathieu, et al.
- 2010
(Show Context)
Citation Context ...lower bounds, data streams, memory checking AMS subject classifications. 68Q17, 68Q25, 68W20, 68W25, 68W32, 62B10 DOI. 10.1137/100816481 1. Introduction. In a recent work, Magniez, Mathieu, and Nayak =-=[21]-=- considered the streaming complexity of language recognition. That is, given a string σ of length n, what is the (randomized) space complexity of a recognizer for a language L that is allowed only seq... |

8 | The one-way communication complexity of gap hamming distance
- Jayram, Kumar, et al.
(Show Context)
Citation Context ...problem. Secondly, the INDEX problem and its variants are important tools in proving a number of lower bounds in the data stream model and on data structures. For instance, the works of Jayram et al. =-=[13]-=- and Kane et al. [14] on the 1-pass space complexity of approximating frequency moments and Lp norm (for 1 ≤ p ≤ 2), of Feigenbaum et al. [9] on lower bounds for various graph streaming problems inclu... |

8 | Cell-probe lower bounds for succinct partial sums
- Pǎtras¸cu, Viola
- 2010
(Show Context)
Citation Context ...istribution. While our focus is on proving this fundamental theorem for its own sake, we note that as an easy application, we obtain a cell-probe lower bound for the well-studied partial sums problem =-=[19]-=-. In the partial sums problem, we are given an input array A[1 . . . n] of bits. We preprocess it into a data structure so that given a query i ∈ [n], we can determine RANK(i) = ∑i k=1 A[k]. We want t... |

6 |
A property of quantum relative entropy with an application to privacy in quantum communication
- Jain, Radhakrishnan, et al.
(Show Context)
Citation Context ...e been better, as it is possible for messages to reveal less information than their length. This issue was considered (in a more general quantum communication setting) by Jain, Radhakrishnan, and Sen =-=[18]-=-, who called this the “privacy tradeoff” for the problem, and showed that a ≥ n/2 O(b) still holds, where a and b now represent information costs. Such an information cost tradeoff opens up interestin... |

5 |
The space complexity of recognizing well-parenthesized expressions in the streaming model: the Index function revisited
- Jain, Nayak
- 2010
(Show Context)
Citation Context ...ue in a way they could not necessarily have been used for the original sequence. Related work. Independently of our work, and concurrent with the first announcement of our results [6], Jain and Nayak =-=[16]-=- presented an alternate proof of the main lower bound result in this paper. The lower bounds in these first announcements were each slightly worse (in incomparable ways) than what we show here. Howeve... |

5 |
Ashwin Nayak. Recognizing well-parenthesized expressions in the streaming model
- Magniez, Mathieu
- 2010
(Show Context)
Citation Context ...y Bar-Yossef et al. [3] as an important step in proving more sophisticated direct sum theorems. 2 Such direct sum arguments are at the heart of a number of tight lower bounds for data stream problems =-=[5,4,10,16]-=-; they are also implicit in several data structure lower bounds by way of communication lower bounds on LOPSIDED-SET-DISJOINTNESS [18]. We now summarize the key previous lower bounds results on INDEX ... |

4 |
Checking and spot-checking of heaps
- Chu, Kannan, et al.
- 2007
(Show Context)
Citation Context ...irectional passes are allowed. A second application is to memory checking, whose study was initiated by Blum et al. [5] and continued by numerous groups including Ajtai [2], Chu, Kannan, and McGregor =-=[11]-=-, Dwork et al. [14], and Naor and Rothblum [23]. The problem, as considered in this paper, is to observe a sequence of N updates and queries to (an implementation of) a data structure and to report wh... |

1 |
Vérification des files de priorité dans le modèle de streaming
- François, Magniez
- 2011
(Show Context)
Citation Context ... ˜ Ω( √ N) passes. To the best of our knowledge, this is the first explicit demonstration of such a strong separation between these two natural data stream models. More recently, François and Magniez =-=[15]-=- have demonstrated that it is also possible to recognize the language pq in O(log 2 N) space using two passes, one in each direction. Given our lower bound in Theorem 4.4, this provides another exampl... |