Pitfalls for Testbed Evaluations of Internet Systems

Date Added: Apr 2010
Format: PDF

Today's open platforms for network measurement and distributed system research, which the authors collectively refer to as testbeds in this papeer, provide opportunities for controllable experimentation and evaluations of systems at the scale of hundreds or thousands of hosts. In this paper, the authors identify several issues with extending results from such platforms to Internet wide perspectives. Specifically, they try to quantify the level of inaccuracy and incompleteness of testbed results when applied to the context of a large-scale Peer-To-Peer (P2P) system. Based on their results, the authors emphasize the importance of measurements in the appropriate environment when evaluating Internet-scale systems.