A Scalable Framework for Heterogeneous GPU-Based Clusters
GPU-based heterogeneous clusters continue to draw attention from vendors and HPC users due to their high energy efficiency and much improved single-node computational performance, however, there is little parallel software available that can utilize all CPU cores and all GPUs on the heterogeneous system efficiently. On a heterogeneous cluster, the performance of a GPU (or a compute node) increases in a much faster rate than the performance of the PCI-Express connection (or the interconnection network) such that communication eventually becomes the bottleneck of the entire system. To overcome the bottleneck, the authors developed a multi-level partitioning and distribution method that guarantees a near-optimal communication volume. They have also extended heterogeneous tile algorithms to work on distributed-memory GPU clusters.