Request-Aware Scheduling for Busy Internet Services
Internet traffic is bursty and network servers are often overloaded with surprising events or abnormal client request patterns. This paper studies scheduling algorithms for interactive network services that use multiple threads to handle incoming requests continuously and concurrently. The authors' investigation with applications from Ask Jeeves search shows that during overloaded situations, requests that require excessive computing resource can dramatically affect the overall system throughput and response time. The most effective method is to manage resource usage at a request level instead of a thread or process level.