Transparent and Flexible Network Management for Big Data Processing in the Cloud

Download Now
Provided by: NEC Laboratories
Topic: Data Management
Format: PDF
The authors introduce FlowComb, a network management framework that helps big data processing applications, such as Hadoop, achieve high utilization and low data processing times. FlowComb predicts application network transfers, sometimes before they start, by using software agents installed on application servers and while remaining completely transparent to the application. A centralized decision engine collects data movement information from agents and schedules upcoming flows on paths such that the network does not become congested. Results on their lab test-bed show that FlowComb is able to reduce the time to sort 10GB of randomly generated data by 35% while changing paths for only 6% of the transfers.
Download Now

Find By Topic