Results 1 - 10 of 1,210

Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing

by Matei Zaharia, Mosharaf Chowdhury, Tathagata Das, Ankur Dave, Justin Ma, Murphy Mccauley, Michael J. Franklin, Scott Shenker, Ion Stoica , 2011
"... We present Resilient Distributed Datasets (RDDs), a distributed memory abstraction that lets programmers perform in-memory computations on large clusters in a fault-tolerant manner. RDDs are motivated by two types of applications that current computing frameworks handle inefficiently: iterative algo ..."
Abstract - Cited by 239 (27 self) - Add to MetaCart
We present Resilient Distributed Datasets (RDDs), a distributed memory abstraction that lets programmers perform in-memory computations on large clusters in a fault-tolerant manner. RDDs are motivated by two types of applications that current computing frameworks handle inefficiently: iterative

Techniques for Efficient In-Memory Checkpointing

by Dirk Vogt, Cristiano Giuffrida, Herbert Bos, Andrew S. Tanenbaum
"... Checkpointing is a pivotal technique in system research, with applications ranging from crash recovery to replay debugging. In this paper, we evaluate a number of in-memory check-pointing techniques and compare their properties. We also present a new compiler-based checkpointing scheme which improve ..."
Abstract - Cited by 1 (1 self) - Add to MetaCart
improves state-of-the-art performance and memory guaran-tees in the general case. Our solution relies on a shadow state to efficiently store incremental in-memory checkpoints, at the cost of a smaller user-addressable virtual address space. Contrary to common belief, our results show that in-memory

Abstractions for In-memory Distributed Computation

by Russell Power, C Russell Power , 2014
"... The recent cloud computing revolution has changed the distributed computing landscape, making the resources of entire datacenters available to ordinary users. This process has been greatly aided by dataflow style frameworks such as MapReduce which expose simple model for programs, allowing for effic ..."
Abstract - Cited by 1 (0 self) - Add to MetaCart
, which help program-mers to write in-memory distributed applications not served well by existing approaches. Piccolo presents a new data-centric programming model for in-memory applications. Unlike data-flow models, Piccolo allows programs running on different machines to share distributed, mutable state

Efficient in-memory top-k document retrieval

by Matthias Petri, Falk Scholer , 2012
"... For over forty years the dominant data structure for ranked document retrieval has been the inverted index. Inverted indexes are effective for a variety of document retrieval tasks, and particularly efficient for large data collection scenarios that require disk access and storage. However, many eff ..."
Abstract - Cited by 9 (2 self) - Add to MetaCart
efficiency-bound search tasks can now easily be supported entirely in-memory as a result of recent hardware advances. In this paper we present a hybrid algorithmic framework for inmemory bag-of-words ranked document retrieval using a self-index derived from the FM-Index, wavelet tree, and the compressed

Phase Reconciliation for Contended In-Memory Transactions

by Neha Narula, Cody Cutler, Eddie Kohler, Robert Morris
"... Multicore main-memory database performance can col-lapse when many transactions contend on the same data. Contending transactions are executed serially—either by locks or by optimistic concurrency control aborts—in order to ensure that they have serializable effects. This leaves many cores idle and ..."
Abstract - Cited by 4 (0 self) - Add to MetaCart
Multicore main-memory database performance can col-lapse when many transactions contend on the same data. Contending transactions are executed serially—either by locks or by optimistic concurrency control aborts—in order to ensure that they have serializable effects. This leaves many cores idle

FPGA-based Multithreading for In-Memory Hash Joins

by Robert J Halstead , Ildar Absalyamov , Walid A Najjar , Vassilis J Tsotras
"... ABSTRACT Large relational databases often rely on fast join implementations for good performance. Recent paradigm shifts in processor architectures has reinvigorated research into how the join operation can be implemented. The FPGA community has also been developing new architectures with the poten ..."
Abstract - Add to MetaCart
requests. In this paper we present the first end-to-end in-memory FPGA hash join implementation. The FGPA uses massive multithreading during the build and probe phases to mask long memory delays, while it concurrently manages hundreds of thread states locally. Throughput results show a speedup between 2x

Rethinking SIMD Vectorization for In-Memory Databases

by Orestis Polychroniou , Arun Raghavan , Kenneth A Ross
"... ABSTRACT Analytical databases are continuously adapting to the underlying hardware in order to saturate all sources of parallelism. At the same time, hardware evolves in multiple directions to explore different trade-offs. The MIC architecture, one such example, strays from the mainstream CPU desig ..."
Abstract - Cited by 2 (0 self) - Add to MetaCart
design of in-memory database operators, as well as the architectural design and power efficiency of hardware, by making simple cores comparably fast to complex cores. This work is applicable to CPUs and co-processors with advanced SIMD capabilities, using either many simple cores or fewer complex cores.

Multithreaded Asynchronous Graph Traversal for In-Memory and Semi-External Memory

by Roger Pearce, Maya Gokhale, Nancy M. Amato
"... Processing large graphs is becoming increasingly important for many computational domains. Unfortunately, many algorithms and implementations do not scale with the demand for increasing graph sizes. As a result, researchers have attempted to meet the growing data demands using parallel and external ..."
Abstract - Cited by 34 (3 self) - Add to MetaCart
), Single Source Shortest Path (SSSP), and Connected Components (CC) for large graphs in shared memory. We present an experimental study applying our technique to both In-Memory (IM) and Semi-External Memory (SEM) graphs utilizing multi-core processors and solid-state memory devices. Our experiments using

Flashback: A Lightweight Extension for Rollback and Deterministic Replay for Software Debugging

by Sudarshan M. Srinivasan, Srikanth K, Christopher R. Andrews, Yuanyuan Zhou - In USENIX Annual Technical Conference, General Track , 2004
"... Unfortunately, finding software bugs is a very challenging task because many bugs are hard to reproduce. While debugging a program, it would be very useful to rollback a crashed program to a previous execution point and deterministically re-execute the "buggy " code region. However ..."
Abstract - Cited by 155 (7 self) - Add to MetaCart
-grained rollback and replay to help debug software. Flashback uses shadow processes to efficiently roll back in-memory state of a process, and logs a process ' interactions with the system to support deterministic replay. Both shadow processes and logging of system calls are implemented in a lightweight

A Scalable Double In-memory Checkpoint and Restart Scheme towards Exascale

by Gengbin Zheng, Xiang Ni, Laxmikant V. Kalé - in Proceedings of the 2nd Workshop on FaultTolerance for HPC at Extreme Scale (FTXS , 2012
"... As the size of supercomputers increases, the probability of system failure grows substantially, posing an increasingly significant challenge for scalability. It is important to provide resilience for long running applications. Checkpoint-based fault tolerance methods are effective approaches at deal ..."
Abstract - Cited by 20 (4 self) - Add to MetaCart
at dealing with faults. With these methods, the state of the entire parallel application is checkpointed to reliable storage. When a failure occurs, the application is restarted from a recent checkpoint. In previous work, we have demonstrated an efficient double in-memory checkpoint and restart fault
Results 1 - 10 of 1,210