Citation Context

...st of the parallel processing systems have only been implemented on smaller scales and leave the details of handling machine failures to the programmer. Bulk Synchronous Programming [17] and some MPI primitives [11] provide higher-level abstractions that make it easier for programmers to write parallel programs. A key difference between these systems and MapReduce is that MapReduce exploits a restricted programming model to parallelize the user program automatically and to provide transparent fault-tolerance. Our locality optimization draws its inspiration from techniques such as active disks [12, 15], where computation is pushed into processing elements that are close to local disks, to reduce the amount of data sent across I/O subsystems or the network. We run on commodity processors to which a small number of disks are directly connected instead of running directly on disk controller processors, but the general approach is similar. Our backup task mechanism is similar to the eager scheduling mechanism employed in the Charlotte System [3]. One of the shortcomings of simple eager scheduling is that if a given task causes repeated failures, the entire computation fails to complete. We fix ...

Citation Context

...ur approach of evaluating persistent queries on the server realizes the same benefit of pushing evaluation to the data that has been previously shown in projects such as Active Disks [19] and Diamond =-=[8]-=-. Salmon et al. [20] have recently proposed views that allow pervasive devices to publish interest in particular sets of objects with peer devices. Views are similar to persistent queries in that they...

Citation Context

...terested in examining. Some problems that appear to be All-Pairs may be algorithmically reducible to a smaller problem via techniques such as data clustering (Google [5]), or early filtering (Diamond =-=[13]-=-). In these two cases, the user’s intent is not AllPairs, but sorting or searching, and thus other kinds of optimizations apply. In the All-Pairs problems that we consider, it is actually necessary to...

Citation Context

...and storage tactics on the fly, to converge to a better strategy over time. Aspects that are, at least in principle, amenable to adaptive optimization at runtime include dataflow graph structure (see =-=[6, 10, 16]-=-), load balancing across partitions (see [5, 26]), data placement (Section 4.2.1), and caching and reuse of derived data (Section 4.2.2). In this paper we lay out some possible optimization strategies...

Citation Context

...abstraction. Note that some problems that appear to be All-Pairs may be algorithmically reducible to a smaller problem via techniques such as data clustering (Google [5]), or early filtering (Diamond =-=[13]-=-). In these three cases, the user’s intent is not All-Pairs, but sorting or searching, and thus other kinds of optimizations apply. In the All-Pairs problems that we consider, it is actually necessary...

Citation Context

... by preprocessing the images and by first focusing the search with an index-based global or region-based search. Additionally, our technique is wellsuited to deployment on active-disk infrastructures =-=[7]-=- for fast searching of non-indexed data. Precision Average Precision-Recall for Ten Categories 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0.2 0.4 0.6 0.8 1 Recall Feedback 0 Feedback 1 Figure 5. The average pr...

Citation Context

...e architectural motivation for FAWN borrows from a long line of work on balanced computing [17], multimicroprocessor systems for image processing [12], supercomputing [7], placing smarts near storage =-=[10, 16, 2]-=-, and creating arrays of cheap components to leverage “disefficiencies” of scale [14]. FAWN takes advantage of a sweet spot in processor efficiency for I/O intensive workloads that do not saturate mod...

Citation Context

...educe the returned results to a more relevant set. This step is very computationally intensive but can be run in parallel on the cloudlets. This step uses early discard, as described by Huston et al. =-=[14]-=-, to increase the selectivity of a result stream. Using a plugin interface, image processing code fragments called filters can be inserted into the result stream. These code fragments allow user-defin...

Citation Context

...lose to sequential disk bandwidth. Recent trends indicate that storage blades with extremely fast processors and large buffer memory are becoming common. The reader is referred to the Diamond project =-=[3]-=- for exhaustive search applications and a storage-level programming model for such applications. 2.2 Search Suspend-and-Resume Upon receiving a real-time request, an ongoing exhaustive search process ...