Prediction of Optimal Parallelism Level in Wide Area Data Transfers

Date Added: Oct 2009
Format: PDF

Wide area data transfers may be a major bottleneck for the end-to-end performance of distributed applications. A practical way of increasing the wide area throughput at the application layer is using multiple parallel streams. Although increased number of parallel streams may yield much better performance than using a single stream, overwhelming the network by opening too many streams may have an inverse effect. The congestion created by excess number of streams may cause a drop down in the throughput achieved. Hence, it is important to decide on the optimal number of streams without congesting the network. Predicting this 'Optimum' number is not straightforward, since it depends on many parameters specific to each individual transfer.