Network Flow Based Resource Brokering and Optimization Techniques for Distributed Data Streaming Over Optical Networks

This paper analyzes the problem of optimizing access to and transport of large remote data through intelligent resource selection and configuration. With the availability of high-speed optical networks, the main problem for remote data access is shifting from having enough network bandwidth to having enough data ready to saturate the network when requested by the application. Network bandwidth is now higher than disk bandwidth and this gives one the possibility of utilizing multiple distributed resources to saturate the network links. The authors are considering two types of scenarios, one where they use only disks as data sources and a more advanced scenario where compute resources in the network can be utilized as caches to increase instantaneous throughput.