Resource management on clouds and grids: challenges and answers

Grids and more recently clouds are distributed system infrastructures that are rapidly gaining popularity among researchers and users. By providing the ability to acquire resources on demand these systems provide elasticity in resource usage as well as a pay-as-you-go opportunity, both of which can lead to a substantial savings for the system users. Appropriate management of resources by the middleware used by service providers is required however for effectively harnessing the power of the underlying distributed resource infrastructure. The problems range from handling resource heterogeneity, providing adequate security during resource access, allocating resources to user requests efficiently as well as effectively scheduling the requests that are mapped to a given resource. This talk will focus on the challenges associated with resource management on grids and clouds and discuss solutions to some of these problems. Particular attention will be paid to resource allocation and scheduling. Existing literature on resource management on grids and clouds describe techniques that are based on the detailed knowledge of local resource management policies as well as user estimates of resource demands for their requests. It is often impractical to assume such a detailed a priroi knowledge of management policies for all the resources will be available to resource brokers in a large and dynamic heterogeneous environment. Moreover, user estimates of resource demands are often error prone. Performing effective resource management "in the dark" and handling such uncertainties associated with the local resource management policies and user estimated resource demands will be discussed. Techniques for providing both user satisfaction by improving the fastness of responses as well as service provider benefits through generation of high revenue will be described.