Composition of Web services has received much interest to support business-to-business or enterprise application integration. On the one side, the business world has developed a number of XML-based standards to formalize the specification of Web services, their flow composition and execution. This approach is primarily syntactical: Web service interfaces are like remote procedure call and the interaction protocols are manually written. On the other side, the Semantic Web community focuses on reasoning about web resources by explicitly declaring their preconditions and effects with terms precisely defined in ontologies. For the composition of Web services, they draw on the goal-oriented inferencing from planning. So far, both approaches have been developed rather independently from each other. We compare these approaches...

Virtual machine distributed computing greatly simplifies the use of widespread computing resources by lowering the level of abstraction, benefiting both resource providers and users. Towards that end our Virtuoso middleware closely emulates the existing process of buying, configuring and using physical machines. Virtuoso's VNET component is a simple and efficient layer two virtual network tool that makes these virtual machines (VMs) appear to be physically connected to the home network of the user while simultaneously supporting arbitrary topologies and routing among them. Virtuoso's VTTIF component continually infers the communication behavior of the application running in a collection of VMs. The combination of overlays like VNET and inference frameworks like VTTIF has great potential to increase the performance, with no user or developer involvement, of existing, unmodified applications by adapting their virtual environments to the underlying computing infrastructure to best suit the applications. We show here how to use the continually inferred application topology and traffic to dynamically control three mechanisms of adaptation, VM migration, overlay topology, and forwarding to significantly increase the performance of two classes of applications, bulk synchronous parallel applications and transactional web ecommerce applications.

Grid computing is emerging as key enabling infrastructure for science. A key challenge for distributed computation over the Grid is the synthesis on-demand of end-toend scientific applications of unprecedented scale that draw from pools of specialized scientific components to derive elaborate new results. In this paper, we outline the technical issues that need to be addressed in order to meet this challenge, including usability, robustness, and scale. We describe Pegasus, a system to generate executable grid workflows given a high-level specification of desired results. Pegasus uses Artificial Intelligence planning techniques to compose valid end-to-end workflows, and has been used in several scientific applications. We also outline our design for a more distributed and knowledge-rich architecture.

Component-based approaches are becoming increasingly popular in the areas of adaptive distributed systems, web services, and grid computing. In each case, the underlying infrastructure needs to address a deployment problem involving the placement of application components onto computational, data, and network resources across a wide-area environment subject to a variety of qualitative and quantitative constraints. In general, the deployment needs to also introduce auxiliary components (e.g., to compress/decompress data, or invoke GridFTP sessions to make data available at a remote site), and reuse preexisting components and data. To provide the flexibility required in the latter case, recently proposed systems such as Sekitei and Pegasus have proposed solutions that rely upon AI planning-based techniques. Although promising, the inherent complexity of AI planning and the fact that constraints governing component deployment often involve non-linear and non-reversible functions have prevented such solutions from generating deployments in resource-constrained situations and achieving optimality in terms of overall resource usage or other cost metrics. This paper addresses both of these shortcomings in the context of the Sekitei system. Our extension relies upon information supplied by a domain expert, which classifies component behavior into a discrete set of levels. This discretization, often justified in practice, permits the planner to identify cost-optimal plans (whose quality improves with the level definitions) without restricting the form of the constraint functions. We describe the modified Sekitei algorithm, and characterize, using a media stream delivery application, its scaling behavior when generating optimal deployments for various network configurations. 1

Web services are becoming important in applications from electronic commerce to application interoperation. While numerous efforts have focused on service composition, service selection among similar services from multiple providers has not been addressed. Such issue is more serious when services are embraced in Grid platforms, which are usually resource-conscious. Experimental results show that our considerations are valid and our preliminary solution works well in our Globus grid network.

In a previous work, we had analyzed the gaps in the prevalent approaches (i.e., Semantic Web Services and WSDLdescribed Web Services) for the problems of modeling, composing, executing, and verifying Web services, and derived challenges for the AI planning community. The challenges were in representation of complex actions, handling of richly typed messages, dynamic object creation and specification of multi-partner interactions. An important question that constantly arose was how the goals for automatic composition would be derived. In this paper, we revisit this issue in the light of new trends in software engineering towards Model Driven Architecture and early deployment of Web service composition solutions. We argue that Web services composition can not be seen as a one-shot plan synthesis problem defined with explicit goals but rather as a continual process of manipulating complex workflows, which requires to solve synthesis, execution, optimization, and maintenance problems as goals get incrementally refined. We then identify additional issues that become important in applying planning techniques.

The optimization problems associated with adaptive and autonomic computing systems are often difficult to pose well and solve efficiently. A key challenge is that for many applications, particularly interactive applications, the user or developer is unlikely or unable to provide either the objective function f, or constraints. It is a key problem encountered broadly in adaptive and autonomic computing. This dissertation argues for using human-driven optimization techniques to solve optimization problems. In particular, it consists of two core ideas. In human-driven specification, we use direct human input from users to pose specific optimization problems, namely to determine the objective function f and expose hidden constraints. Once we have a well-specified problem, we are left with the need to search for a solution in a very large solution space. In human-driven search, we use direct human input to guide the search for a good solution, a valid configuration x that optimizes f(x). My research happens in three contexts. The main context is the Virtuoso system for utility and grid computing based on virtual machines (VMs) interconnected with overlay

Over the last fifteen years we have seen a tremendous increase in computer and network speeds and performance, resulting in the emergence of a new computing paradigm, wide-area distributed computing. However, the full potential of wide-area distributed computing has not been exploited, primarily due to the challenges involved in developing applications for such environments. The wide fluctuations in available resources and inherent heterogeneity of distributed environments require adaptation in each application. Such custom adaptation is exceedingly complex as the application requirements, computational and network resources can vary over time resulting in adaptation mechanisms and control not being common on today’s applications. We believe that one way of realizing the full potential of wide-area distributed computing is by building a virtual execution environment consisting of operating system level virtual machines connected by virtual net-works. Virtualization technology such as Virtual Machine Monitors (VMMs) can greatly simplify distributed computing by lowering the level of abstraction from traditional units of work, such as jobs, processes, or RPC calls to that of a raw machine. Such execution environments make possible low-level, application-, developer-, and user-independent

Virtual machine based distributed computing greatly simplifies and enhances adaptive / autonomic computing by lowering the level of abstraction, benefiting both resource providers and users. We are developing Virtuoso, a middleware system for virtual machine shared resource computing (e.g. grids) that provides numerous advantages and overcomes many obstacles a user faces in using shared resources for deploying distributed applications. A major hurdle for distributed applications to function in such an environment is locating, reserving/scheduling and dynamically adapting to the appropriate communication and computational resources so as to meet the applications ’ demands, limited by cost constraints. Resources can be very heterogeneous, especially in wide area or shared infrastructures, and their availability is also highly dynamic. To achieve such automated adaptation, one must first learn about the various demands and properties of the distributed application running inside the VMs. My thesis is that it is feasible to infer the applications ’ demands and behavior, to a significant degree, without actually knowing much about the application or its operating system; I have investigated and demonstrated numerous novel techniques to infer many useful properties of parallel applications, in an automated fashion. Throughout my work

Advances in high-performance computing have led to the broad use of computational studies in everyday engineering and scientific applications. A single study may require thousands of computational experiments, each corresponding to individual runs of simulation software with different parameter settings; in complex studies, the pattern of parameter changes is complex and may have to be adjusted by the user based on partial simulation results. Unfortunately, existing tools have limited high-level support for managing large ensembles of simultaneous computational experiments. In this paper, we present a system architecture for interactive computational studies targeting two goals. The first is to provide a framework for high-level user interaction with computational studies, rather than individual experiments; the second is to maximize the size of the studies that can be performed at close to interactive rates. We describe a prototype implementation of the system and demonstrate performance improvements obtained using our approach for a simple model problem. 1