Stony Brook Computer Science Dept.
The authors introduce LiPS, a new cost-efficient data and task co-scheduler for MapReduce in a cloud environment. By using linear programming to simultaneously co-schedule data and tasks, LiPS helps to achieve minimized dollar cost globally. They evaluated LiPS both analytically and on Amazon EC2. Results are significant. LiPS saved up to 81% of the actual dollar costs when compared with both the Hadoop default and the more performant delay scheduler, while also allowing users to fine-tune the cost-performance tradeoff. LiPS presents today's most cost-efficient scheduler on and should be deployed when constraints on overall make span are flexible.