Availability Modeling and Assurance of Map-reduce Computing

Abstract

This thesis proposes a new analytical model to evaluate the availability of Map-Reduce computing on a Hadoop platform. Map-Reduce computing is represented by a queueing model in order to trace flow of Map-Reduce jobs of their arrivals and departures in the course of computation. The objective of this analytical model is to evaluate the probability for a Map-Reduce computation to be available at an instant of time, referred to as availability. The set of variables taken into account in this model lists the number of Map-Reduce jobs, the number of servers (or referred to as the worker nodes in this thesis) engaged, along with a few constants such as job arrival/completion rates and the worker node failure/repair rates. The proposed model provides a comprehensive yet fundamental basis to assure and ultimately optimize the design of Map-Reduce computing in terms of availability with reference to its performance in a simultaneous manner. Parametric simulations have been conducted and demonstrated efficacy of the proposed model in assessing the availability and the cost for achieving the availability with respect to throughput as well as turnaround time.