By defining standardized protocols for discovering, accessing, monitoring, and managing remote computers, storage systems, networks, and other resources, Grid technologies make it possible—in principle—to allocate resources to applications dynamically, in an on-demand fashion [1]. However, while Grids offer users access to many diverse and powerful resources, they do little to ensure that once a

This paper describes a virtual le system that allows data to be transferred on demand between storage and compute servers for the duration of a computing session. The solution works with unmodi ed applications (even commercial ones) running on standard operating systems and hardware. The virtual le system employs software proxies to broker transactions between standard NFS clients and servers; the proxies are dynamically con gured and controlled bycomputational grid middleware. The approach has been implemented and extensively exercised in the context of the Purdue University Network Computing Hubs, an operational computing portal that has more than 1,500 users across 24 countries. Results show that the virtual le system performs well in comparison to native NFS: performance analyses show that the proxy incurs mean overheads of 1 % and 18 % with respect to native NFS for a singleclient execution of the Andrew benchmark in two representative computing environments, and that the average overhead for eight clients can be reduced to within 1 % of native NFS with concurrent proxies. 1.

Management of complex, distributed, and dynamically changing job executions is a central problem in computational Grids. These executions often span multiple heterogeneous resources, cross administrative domains, and need to adjust to the changing resource availability to leverage opportunities and account for failures or policy induced failures. The executions themselves are

This paper presents a data management solution which allows fast Virtual Machine (VM) instantiation and efficient run-time execution to support VMs as execution environments in Grid computing. It is based on novel distributed file system virtualization techniques and is unique in that: 1) it provides on-demand access to VM state for unmodified VM monitors; 2) it supports user-level and write-back disk caches, per-application caching policies and middleware-driven consistency models; and 3) it supports the use of meta-data associated with files to expedite data transfers. The paper reports on its performance in a WAN setup using VMware-based VMs. Results show that the solution delivers performance over 30 % better than native NFS and can bring application-perceived overheads below 10 % relatively to a local disk setup. The solution also allows a VM with 1.6GB virtual disk and 320MB virtual memory to be cloned within 160 seconds when it is first instantiated (and within 25 seconds for subsequent clones). 1.

This paper describes techniques for establishing private distributed file system sessions for computational grids. These techniques build on previous work on proxy-based virtualization of Network File Systems (NFS); novel in this paper are the support for multiple proxies, encrypted tunneling via the Secure Shell protocol (SSH), and user-level, proxy-based client disk caching. These techniques can be implemented through middleware-controlled creation of proxies and tunnels, and can be integrated with existing grid infrastructures. Results from performance analyses conducted in wide- and local-area setups show that the application-perceived overhead of this solution is small (10% or less) for the compute-intensive application SPECseis. For the remaining studied applications -- Andrew, virtual machine (VM) boot, and VM resume -- results show that efficient caching is important to achieve good performance. With VM state cached in disk by a client file system proxy, subsequent instantiations of a resumable non-persistent classic VM take 6 to 16 seconds.

Abstract. This paper presents a data management solution which allows fast Virtual Machine (VM) instantiation and efficient run-time execution to support VMs as execution environments in Grid computing. It is based on novel distributed file system virtualization techniques and is unique in that: (1) it provides on-demand cross-domain access to VM state for unmodified VM monitors; (2) it enables private file system channels for VM instantiation by secure tunneling and session-key based authentication; (3) it supports user-level and write-back disk caches, per-application caching policies and middleware-driven consistency models; and (4) it leverages application-specific meta-data associated with files to expedite data transfers. The paper reports on its performance in wide-area setups using VMware-based VMs. Results show that the solution delivers performance over 30 % better than native NFS and with warm caches it can bring the application-perceived overheads below 10 % compared to a local-disk setup. The solution also allows a VM with 1.6 GB virtual disk and 320 MB virtual memory to be cloned within 160 seconds for the first clone and within 25 seconds for subsequent clones.

Over the last fifteen years we have seen a tremendous increase in computer and network speeds and performance, resulting in the emergence of a new computing paradigm, wide-area distributed computing. However, the full potential of wide-area distributed computing has not been exploited, primarily due to the challenges involved in developing applications for such environments. The wide fluctuations in available resources and inherent heterogeneity of distributed environments require adaptation in each application. Such custom adaptation is exceedingly complex as the application requirements, computational and network resources can vary over time resulting in adaptation mechanisms and control not being common on today’s applications. We believe that one way of realizing the full potential of wide-area distributed computing is by building a virtual execution environment consisting of operating system level virtual machines connected by virtual net-works. Virtualization technology such as Virtual Machine Monitors (VMMs) can greatly simplify distributed computing by lowering the level of abstraction from traditional units of work, such as jobs, processes, or RPC calls to that of a raw machine. Such execution environments make possible low-level, application-, developer-, and user-independent

management architecture for computational grids. The design is based on two key realizations. One is that resource management involves a sequence of tasks that is best handled by a pipeline. As shown in the paper, this approach results in a scalable architecture for decentralized scheduling. The other realization is that static aggregation of resources for improved scheduling is inadequate in wide-area computing environments because the needs of users and jobs change with both, location and time. The described architecture addresses this problem by dynamically aggregating resources in a manner that continuously optimizes system response. This is accomplished by way of an active yellow pages directory that allows aggregation constraints to be (re)defined on the fly. An initial prototype of the active yellow pages service has been deployed in the PUNCH network computing environment. Experiences with the production PUNCH system and preliminary results from controlled experiments indicate that the active yellow pages service performs well.

Abstract: E-Science often requires access to remote Grid computing platforms. Current authorization systems on these remote systems have largely based decisions solely on the identity of the submitter-- the job is permitted to execute on the local resource if the job originates from an authenticated and authorized end-user. The problem with this approach is that there is no consideration to what the job will/should do when executed, so an errorful or malicious job-- even from what purports to be a trusted user-- can create significant damage before an operator notices and can kill or suspend the job. This paper presents a novel end-to-end job execution framework in which the properties (behavior) of the job are taken into account for the authorization decision. Experimental results show the duration to perform the authorization and to establish a subsequent restrictive execution context is sufficiently low – our observed overhead of 253.1 ms on commodity hardware is an acceptable cost for most Grid applications to pay to achieve this more secure execution environment. 1.

Abstract. The In-VIGO approach to Grid-computing relies on the dynamic establishment of virtual grids on which application services are instantiated. In-VIGO was conceived to enable computational science to take place In Virtual Information Grid Organizations. Having its first version deployed on July of 2003, In-VIGO middleware is currently used by scientists from various disciplines, a noteworthy example being the computational nanoelectronics research community