Working Sets, Cache Sizes, and Node Granularity Issues for Large-Scale Multiprocessors

The distribution of resources among processors, memory and caches is a crucial question faced by designers of large-scale parallel machines. If a machine is to solve problems with a certain data set size, should it be built with a large number of processors each with a small amount of memory, or a smaller number of processors each with a large amount of memory? How much cache memory should be provided per processor for cost-effectiveness? And how do these decisions change as larger problems are run on larger machines? In this paper, the authors explore the above questions based on the characteristics of five important classes of large-scale parallel scientific applications.