Exploitation of Long-Range Dependence in Internet Traffic for Resource and Traffic Management

A long-held belief regarding the scaling behavior---self-similar or Long-Range Dependence (LRD) of Internet traffic is that the scaling behavior has adverse impacts on the Internet. In particular, it causes the queue length tend to infinity and hence severe packet drops. The scaling property means much more bursty traffic which consists of concentrated periods of high activity and low activity at a wide range of time scales, and this burstiness adversely affects resource management and degrades the overall Internet performance. In this thesis we study the scaling behavior of the Internet traffic from a totally different perspective, i.e., we aim to take advantage of the self-similar/LRD property of the Internet traffic for the sake of resource and traffic control. Since self-similar/LRD implies the existence of nontrivial correlation structure at any time scales, accurate prediction can be faithfully achieved and the prediction results can be used to do resource and traffic control. In light of this, first, in order to make a thorough understanding of the scaling behavior, we propose a hierarchical model that has an one-on-one correspondence to the protocols in the protocol hierarchy of IP networks and give insights on how, and to what extent, the user/protocol behavior in each protocol layer contributes to scaling properties. Then based on the understanding, we propose: (1) Predictive Active Queue Management (PAQM) to stabilize the queue lengths at routers based on the prediction of future incoming traffic; (2) TCP with Traffic Prediction (TCP-TP) to fasten the process of reaching the optimal operational point in the congestion control based on the prediction of attainable throughput one or two RTTs ahead in the future; (3) Three theoretically grounded algorithms: prediction, reconstruction and interpolation to do proactive non-intrusive end-to-end measurement. We implement them in both user and kernel level. Real Internet experiments are done based on the user-level implementation; Last, for the techniques of passive Internet traffic sampling, we provide an in-depth analytical study of the sampling techniques for the scaling-behaved Internet traffic. To overcome the adverse impacts of the scaling property (heavy-tailedness), we propose a Biased Systematic Sampling (BSS) method to capture both the first and second order statistics of the Internet traffic