Citation Context

... run virtual appliances [55], or provide processes with access to privileged instructions [7]. Similar to IX, Arrakis uses hardware virtualization to separate the I/O dataplane from the control plane =-=[50]-=-. IX differs in that it uses a full Linux kernel as the control plane; provides three-way isolation between the control plane, networking stack, and application; and proposes a dataplane architecture ...

Citation Context

...iderably less complex to understand, develop and deploy. In this work, we consider only latency tails that result from in-network interference. Other work mitigates hostbased sources of latency tails =-=[14, 23, 30, 32, 36]-=-. 0 100 200 300 400 500 Time since start [sec] -400 -200 0 200 400 600 C lo ck of fs et [µ s] ptpd only ptpd with Hadoop (a) Timeline of PTP synchronization offset. 0 500 1000 1500 2000 Request latenc...

Citation Context

...nt instances of ClickOS VMs to reach line rates. IX [11] is a dataplane OS designed to run with Linux controlplane on virtualized environments, with latest hardware supports and optimization. Arrakis =-=[43]-=- uses virtualization to allow direct accesses to the hardware for the raw performance while keeping the same control semantics enforced by the OS. Our work is complimentary to them because NBA is an a...

Citation Context

...9], Intel’s DPDK [10], OpenOnload [22], PF RING [16], PacketShader I/O [8] and Packet MMAP [14]. Some other works go further than a “simple” Linux module bypassing the kernel, like IX [5] and Arrakis =-=[17]-=-. We won’t consider them as, for our purpose, they only offer fast access to raw packets, but in a more protected way than the others I/O frameworks, using virtualization techniques, and they were not...

Citation Context

... the OS kernel for I/O operations in the common case [20]. Our experience shows that we can expect other I/O stacks, such as that of the network, to achieve comparable latency and throughput benefits =-=[21]-=-. 2 Inefficiencies in Storage Stacks To illustrate the overhead of today’s OS storage stacks when conducting small writes with persistence requirements, we conduct an experiment, where we execute smal...

Citation Context

...ew stages with new functionality using filter drivers common in most OSes [18, 34, 38], or appliances over the network [48]. Controlling or programming how IOs flow through this stack is hard if not impossible, for tenants and service providers alike. Once an IO enters the system, the path to its endpoint is pre-determined and static. It must pass through all stages on the way to the endpoint. A new stage with new functionality means a longer path with added latency for every IO. As raw storage and networking speeds improve, the length of the IO stack is increasingly becoming a new bottleneck [43]. Furthermore, the IO stack stages have narrow interfaces and operate in isolation. Unlocking functionality often requires coordinating the functionality of multiple such stages. These reasons lead to applications running on a general-purpose IO stack that cannot be tuned to any of their specific needs, or to one-off customized implementations that require application and system rewrite. This paper’s main contribution is experimenting with applying a well-known networking primitive, routing, to the storage stack. IO routing provides the ability to dynamically change the path and destination of...

Citation Context

...technology, in the form of flash solidstate drives (flash SSDs), is steadily replacing prior storage technology based on the value of affordable microsecond-level random access memory. However, the user-experienced performance of these SSDs has yet to reach its full potential since the time spent in the software portion of the I/O path has been magnified [5, 4, 7]. These software overheads were initially noticed in studies of next-generation memory technologies, such as PCM and STT-MRAM, which are expected to give nanosecond-scaled latencies, and ignited several efforts to address the problem [3, 9, 8, 6]. Yet, these studies do not apply well when considering flash SSDs because the latency of flash memory technology scales at microseconds. This large latency requires a significant amount of parallelism in both SSDs and host systems in order to benefit the users. Flash memory latency, whether in nanoseconds or milliseconds, makes it difficult to decide whether to yield the CPU or not when the CPU awaits I/O completion since the scheduling delay can cause a non-negligible impact on performance. Blocking the CPU (i.e., polling [9]) would avoid such delays but would be at the cost of sacrificing p...

Citation Context

...uch APIs and programming models will make it easier for applications to benefit from the performance and scalability advantages of IX. 7 Related Work We organize the discussion topically, while avoiding redundancy with the commentary in §2.3. Hardware virtualization: Hardware support for virtualization naturally separates control and execution functions, e.g., to build type-2 hypervisors [10, 33], run virtual appliances [55], or provide processes with access to privileged instructions [7]. Similar to IX, Arrakis uses hardware virtualization to separate the I/O dataplane from the control plane [50]. IX differs in that it uses a full Linux kernel as the control plane; provides three-way isolation between the control plane, networking stack, and application; and proposes a dataplane architecture that optimizes for both high throughput and low latency. On the other hand, Arrakis uses Barrelfish as the control plane [6] and includes support for IOMMUs and SR-IOV. Library operating systems: Exokernels extend the endto-end principle to resource management by implementing system abstractions via library operating systems linked in with applications [19]. Library operating systems often run as ...

Citation Context

...ork exist, in addition to the general areas discussed above. In the area of latency analyses of network stacks, [33] analyses the latency breakdown of memcached, in order to 3 46 2016 USENIX Annual Technical Conference USENIX Association understand how to fulfill the quality-of-service objectives of multiple workloads. Their measurements show much higher processing delays than ours, and their focus is not on improving the OS network service. Also, [4] reports a one-way latency of 24 µs with Linux TCP/IP, which is twice what we measure in Section 3. Our results are similar to those reported in [47], taking into account that they use UDP instead of TCP. In the area of API extensions, many UNIX variants (including Linux) implement a sendfile() syscall that transmits a regular file from the buffer cache directly into a TCP socket without a memory copy or multiple context switches. The Linux sendmmsg() and recvmmsg() syscalls support passing multiple messages to and from the kernel at a time. However, they do not allow batching across different descriptors. In late 2015, the Linux kernel connection multiplexer (KCM) [7] was proposed. It enables applications to use message-based interfaces, ...

Citation Context

...d disaggregated memory for implementing snapshot isolation and scalable transactions. Crail’s disaggregated architecture builds on similar concerns of providing fast access to remote resources. However, in contrast to those existing works, Crail provides a comprehensive solution to accessing memory and storage across a cluster, both local and disggregated. For instance, Crail naturally exploits any mix of local and remote resources at a fine granular manner through horizontal tiering. Finally, there has been a body of work on improving the I/O efficiency within a single machine, most recently [23, 2, 5, 13]. These efforts are orthogonal to Crail as Crail permits different user-level I/O techniques to be integrated into its distributed I/O architecture through the storage tier interface. 5 Conclusion We presented Crail, an I/O architecture tailored to best integrate fast networking and storage hardware into distributed data processing platforms. Crail is based on a distributed data store acting as a fast I/O bus for platform specific I/O modules. Crail not only takes full advantage of the capabilities of modern hardware in terms of zero-copy or asynchronous I/O, but it also changes the way local ...