High Performance Parallel Computing With Cloud and Cloud Technologies

This paper presents the experiences in applying, developing, and evaluating cloud and cloud technologies. First, this paper presents the experience in applying Hadoop and DryadLINQ to a series of data/compute intensive applications and then compares them with a novel MapReduce runtime developed by them, named CGL-MapReduce, and MPI. Preliminary applications are developed for particle physics, bioinformatics, clustering, and matrix multiplication. This paper identifies the basic execution units of the MapReduce programming model and categorizes the runtimes according to their characteristics. MPI versions of the applications are used where the contrast in performance needs to be highlighted. This paper discusses the application structure and their mapping to parallel architectures of different types, and look at the performance of these applications.