A Unified Modeling Framework for Distributed Resource Allocation of General Fork and Join Processing Networks

This paper addresses the problem of distributed resource allocation in general fork and join processing networks. The problem is motivated by the complicated processing requirements arising from distributed data intensive computing. In such applications, the underlying data processing software consists of a rich set of semantics that include synchronous and asynchronous data fork and data join. The different types of semantics and processing requirements introduce complex interdependence between various data flows within the network. The authors study the distributed resource allocation problem in such systems with the goal of achieving the maximum total utility of output streams. Past research has dealt with networks with specific types of fork/join semantics, but none of them included all four types.