Citation Context

...totically optimal. B. Edit distance The problem of determining the minimum number of edits (insertions or deletions) required to transform one string into another has a long history in the literature =-=[6, 11]-=-. Orlitsky [22] shows that the amount of communication Cα̂(x, y) necessary to reconcile two strings x and y (of lengths |x| and |y| respectively) that are known to be at most α̂-edits apart is at most...

Citation Context

...ents into an equivalent characteristic polynomial, so that the problem of set reconciliation is reduced to an equivalent problem of rational function interpolation, much like in Reed-Solomon decoding =-=[18]-=-. The resulting algorithm requires one message of roughly bm bits of communication and bm3 computation time to reconcile two sets that differ in m entries. The approach can be improved to expected bm ...

Citation Context

...eads [4] and reconstruction of protein sequences from K-peptides [28]. This idea has even shown up in computational linguistics, where it was used to learn transformations on varying-length sequences =-=[27]-=-. In a simple formal statement of the unique string decoding problem, one is given a string s ∈ Σ∗ over the alphabet Σ. The string is considered uniquely decodable if there is no other string s′ ∈ Σ∗ ...

Citation Context

...two hosts seek to reconcile remote strings that differ in a fixed number of unknown edits, using a minimum amount of communication. A similar problem is faced in cryptography through fuzzy extractors =-=[8]-=-, which can be used to match noisy biometric data to encrypted baseline measurements in a secure fashion. Within a biological context, this problem has common roots with the sequencing of DNA from sho...

Citation Context

...ication points can be added to probabilistic check the result). The merged shingle indices, which can be determined independently of the reconciliation, can be encoded with any standard rateless code =-=[3, 17, 29]-=-, and the two rateless streams can be combined by considering them inputs to yet a third rateless encoding. VI. CONCLUSION We have provided a linear-time algorithm for determining whether a given stri...

Citation Context

...ication points can be added to probabilistic check the result). The merged shingle indices, which can be determined independently of the reconciliation, can be encoded with any standard rateless code =-=[3, 17, 29]-=-, and the two rateless streams can be combined by considering them inputs to yet a third rateless encoding. VI. CONCLUSION We have provided a linear-time algorithm for determining whether a given stri...

Citation Context

...xpect each node in the de Bruijn graph of length l shingles to have only one outgoing edge (implying unique decodability) if l ≤ n+ 1 + W (− ln(p)p−n) ln p , (5) where W (·) is the Lambert W function =-=[5]-=-. When n goes to infinity, then (5) is O(log(n)), meaning that logarithmically sized shingles should avoid communicationally expensive merges. Thus, when the two strings are composed of random iid bit...

Citation Context

...s time complexity O(n|Σ|3) and space complexity Θ(|Σ|3). A. Approach Two principal approaches have been put forth for deciding unique string decodability. The first is due to Pevzner [25] and Ukkonen =-=[32]-=-, who characterized the type of strings that have the same collection of shingles. This approach can be used to generate a simple unique decodability tester whose naive worst-case running time on stri...

Citation Context

...α log |y| (log |y|+ log log |y|+ log(1/ǫ) + logα) bits of communication. Other approaches include those of Evfimievski [10] for small edit distances, Suel [30] based on deltacompression, and Tridgell =-=[31]-=- which presents the computationally efficient (but potentially communicationally inefficient) rsync protocol. C. Reconciliation Another natural approach to the α-edits problem involves the utilization...

Citation Context

...e data with minimum communication. a) Set reconciliation: The problem of set reconciliation seeks to reconcile two remote sets SA and SB of b-bit integers using minimum communication. The approach in =-=[20]-=- involves translating the set elements into an equivalent characteristic polynomial, so that the problem of set reconciliation is reduced to an equivalent problem of rational function interpolation, m...

Citation Context

...efficiently reconstructing a string from a given encoding is fundamental to a broad range of settings. In the information theory world, this is related to the α-edits or string reconciliation problem =-=[4, 22]-=-, wherein two hosts seek to reconcile remote strings that differ in a fixed number of unknown edits, using a minimum amount of communication. A similar problem is faced in cryptography through fuzzy e...

Citation Context

...efficiently reconstructing a string from a given encoding is fundamental to a broad range of settings. In the information theory world, this is related to the α-edits or string reconciliation problem =-=[4, 22]-=-, wherein two hosts seek to reconcile remote strings that differ in a fixed number of unknown edits, using a minimum amount of communication. A similar problem is faced in cryptography through fuzzy e...

Citation Context

...an expect a unique decoding for substrings of identically distributed, independent random bits as long as the substrings are roughly logarithmic in the size of the overall decoded string. The work in =-=[9]-=- also provides evidence of a high probability of unique decoding for logarithmically sized substrings, and includes generalizations to non-binary and even non-uniformly random characters for the strin...

Citation Context

...ication points can be added to probabilistic check the result). The merged shingle indices, which can be determined independently of the reconciliation, can be encoded with any standard rateless code =-=[3, 17, 29]-=-, and the two rateless streams can be combined by considering them inputs to yet a third rateless encoding. VI. CONCLUSION We have provided a linear-time algorithm for determining whether a given stri...

Citation Context

...algorithm [14] has time complexity O(n|Σ|3) and space complexity Θ(|Σ|3). A. Approach Two principal approaches have been put forth for deciding unique string decodability. The first is due to Pevzner =-=[25]-=- and Ukkonen [32], who characterized the type of strings that have the same collection of shingles. This approach can be used to generate a simple unique decodability tester whose naive worst-case run...

Citation Context

...bits of communication and bm3 computation time to reconcile two sets that differ in m entries. The approach can be improved to expected bm communication and computation through the use of interaction =-=[19]-=- and generalized to multisets and to arbitrary error-correcting codes [12]. b) String reconciliation: A string σ can be transformed into a multiset S through shingling, or collecting all contiguous su...

Citation Context

...ncludes generalizations to non-binary and even non-uniformly random characters for the strings. This is extended in [2] to characterize the number of decodings for a given collection of shingles, and =-=[26]-=- considers decoding from regularly gapped collections of substrings in a DNA sequencing framework. Finally, [21] considers an information-theoretic capacity of the sequencing problem, and presents a g...

Citation Context

... that does not need to know the number of edits in advance and requires at most 2α log |y| (log |y|+ log log |y|+ log(1/ǫ) + logα) bits of communication. Other approaches include those of Evfimievski =-=[10]-=- for small edit distances, Suel [30] based on deltacompression, and Tridgell [31] which presents the computationally efficient (but potentially communicationally inefficient) rsync protocol. C. Reconc...

Citation Context

...differ in m entries. The approach can be improved to expected bm communication and computation through the use of interaction [19] and generalized to multisets and to arbitrary error-correcting codes =-=[12]-=-. b) String reconciliation: A string σ can be transformed into a multiset S through shingling, or collecting all contiguous substrings of a given length, including repetitions. For example, shingling ...

Citation Context

...(y)⌉ ≤ log (( |y|+ α̂ α̂ )) + 3 log(α̂), although he leaves an efficient one-way protocol as an open question. The literature includes a variety of proposed protocols for this problem. Cormode et al. =-=[7]-=- propose a hash-based approach that requires a known bound α̂ on edits between x and y (assuming, without loss of generality, that y is the longer string) and communicates at most 4α log( 2|y| α ) log...

Citation Context

...s language, and instead we have exhibited an equivalent NFA with Θ(|Σ|3) states. There has also been work on the probability of a collection of shingles having a unique reconstruction. The authors in =-=[1]-=- show that one can expect a unique decoding for substrings of identically distributed, independent random bits as long as the substrings are roughly logarithmic in the size of the overall decoded stri...

Citation Context

...is the longer string) and communicates at most 4α log( 2|y| α ) log(2α̂) +O ( α log |y| log log(|y|) ln 11−ǫ ) (1) bits to reconcile the strings with probability of failure ǫ. Orlitsky and Viswanthan =-=[24]-=- propose a interactive protocol that does not need to know the number of edits in advance and requires at most 2α log |y| (log |y|+ log log |y|+ log(1/ǫ) + logα) bits of communication. Other approache...

Citation Context

...er of edits in advance and requires at most 2α log |y| (log |y|+ log log |y|+ log(1/ǫ) + logα) bits of communication. Other approaches include those of Evfimievski [10] for small edit distances, Suel =-=[30]-=- based on deltacompression, and Tridgell [31] which presents the computationally efficient (but potentially communicationally inefficient) rsync protocol. C. Reconciliation Another natural approach to...

Citation Context

...tring character. • Line 9. The key observation here is that the graph is necessarily sparse, since any node with more than two parents or children necessarily renders the graph not uniquely decodable =-=[15]-=-. As such, the graph can be stored as an adjacency list so that this line represents a constant time operation for each string character. • Lines 10-19. We maintain a stack onto which vertices are pus...

Citation Context

...ne measurements in a secure fashion. Within a biological context, this problem has common roots with the sequencing of DNA from short reads [4] and reconstruction of protein sequences from K-peptides =-=[28]-=-. This idea has even shown up in computational linguistics, where it was used to learn transformations on varying-length sequences [27]. In a simple formal statement of the unique string decoding prob...

Citation Context

...bservation that the set of uniquely decodable strings form a regular language [15]. With this observation, it is possible to produce a deterministic finite state machine on exp(Ω(|Σ| log |Σ|)) states =-=[16]-=- and a non-deterministic one on O(|Σ|3) states [14]. The DFA is prohibitively expensive to construct explicitly, while the NFA may be simulated in time O(n|Σ|3) and space Θ(|Σ|3). In this work, we pre...

Citation Context

... in [2] to characterize the number of decodings for a given collection of shingles, and [26] considers decoding from regularly gapped collections of substrings in a DNA sequencing framework. Finally, =-=[21]-=- considers an information-theoretic capacity of the sequencing problem, and presents a greedy algorithm for reconstruction that is asymptotically optimal. B. Edit distance The problem of determining t...

Citation Context

... same connected component if and only if [ia, ja]∩ [ib, jb] 6= ∅. This check is a constant-time operation per character. V. STRING RECONCILIATION We next present the string reconciliation protocol in =-=[13]-=- as a specific example where our online unique decodability algorithm is applicable. This specific protocol is a refinement of a shingling approach in [1], and is further based on a transformation to ...