Latency Measurement, Modelling and Management for Interactive Remote Rendering

Ph. D. ThesisInteractive Remote Rendering (IRR) systems enable computationally intensive rendering tasks to be offloaded to powerful remote servers, while permitting real-time user interaction. By streaming only images from the server to the client, these systems solve many issues, but can be adversely affected by Interaction Latency (IL). This thesis explores the use of keyboard-based user interaction prediction as a potential method for reducing IL. Specifically, the following questions are addressed: What is the effect of prediction on IL? How can we model and simulate latency? How can we measure IL when prediction is used? What is the optimal number of predictions ahead required to minimise latency? On which side of the network should prediction be performed? The literature describes a few cases of prediction being used in IRR systems but there exists a lack of knowledge pertaining to the development, integration and measurement of prediction into such systems. Initial investigation identified a lack of robust techniques for simulating and measuring latency in IRR systems, especially those employing prediction. A latency model is introduced, and a simulator is developed, demonstrating results comparable to the real-world. Latency simulation is shown to be accurate and is integrated into a “IRR simulator platform”, permitting the exploration of the above research questions. As a result, two novel latency measurement techniques are presented. A prediction module is then developed and used in conjunction with the simulator platform. Results show that IL can be substantially reduced but predicting too far ahead negatively impacts IL, while less interaction history is found to result in lower mean IL. Finally, Client-Side Prediction was found to be more favourable for IL with respect to the amount of interaction history used, while Server-Side Prediction is shown to facilitate lower IL when predicting more than one step ahead. The results and tools presented in this thesis should prove useful for future exploration of PIRR systems.EPSR