Demystifying Internet traffic

Understanding Internet traffic is important in a number of different scenarios. Internet service providers (ISPs) would like to understand traffic demand at different points in the network while taking capacity planning measures for the future. Router manufacturers would benefit from understanding traffic characteristics while performing queue managements studies. Similarly, existing Internet traffic interacts with and influences the performance of any new deployed service that we wish to test on the Internet. In order to assist with these scenarios, we propose Internet-like traffic generation for local testbeds with the ability to project traffic demands into alternate scenarios. Achieving this goals requires addressing challenges. For instance, traffic mix changes over time and application semantics keep evolving. Moreover, Internet traffic exhibits a statistically rich structure at multiple timescales. The cornerstone of this thesis is the contribution of a tool called Swing which addresses the above challenges. It is the first realistic and responsive traffic generator that can reproduce observed burstiness properties in Internet traffic at multiple time scales. We validate Swing using publicly available traffic traces and then present two of the many different uses it enables. We demonstrate that it is important to consider background traffic for evaluation of distributed systems. The fact that Internet traffic is bursty means that it interacts with application traffic in complex ways, and the outcome cannot be predicted a- priori. We propose that system evaluation should proceed by subjecting the application to realistic traffic settings, and more importantly, to a variety of different scenarios. In particular, we demonstrate that some of the common techniques the community has been using, including Poisson and CBR (constant bit rate) traffic, can lead to misleading conclusions. Finally, we demonstrate the utility of Swing to project traffic demands into the future. We start by exposing the limitations of existing techniques due to their inability to understand/model evolution. Thus, we propose a simple mechanism that network administrators could use to generate traffic demands based on intuitively projected what-if scenario