Cutting MapReduce Cost With Spot Market
Spot market provides the ideal mechanism to leverage idle CPU resources and smooth out the computation demands. Unfortunately, few applications can take advantage of spot market because they cannot handle sudden terminations. The authors describe Spot Cloud MapReduce, the first MapReduce implementation that can fully take advantage of a spot market. Even if a massive number of nodes are terminated regularly due to a price increase, Spot Cloud MapReduce can still make computation progress. They show experimentally that it performs well and it has very little overhead.