Scheduling Workflows in Multi-Cluster Environments
Scientific applications modeled as workflows can exhibit both task and data parallelism. Scheduling these workflows in a multi-cluster environment is challenging due to the large number of task mapping possibilities. Therefore, several heuristics have been proposed over the last years to address such a problem. A key limitation of existing heuristics for multi-cluster environments is that individual tasks are mapped onto single resources, which limits the resource options to reduce the time to the complete workflow executions. This paper introduces the Multi-Cluster Allocation-Heterogeneous Earliest Finish Time (MCA-HEFT) heuristic, which deploys single parallel tasks of a workflow into multiple clusters and schedules them accordingly.