Modeling and Optimizing User Experience for Cloud Mobile 3D Applications

Cloud gaming architecture has been proposed and deployed in large-scale com- mercial systems in recent years. It renders and encodes video views on cloud servers, with the resulting video streamed through network to end devices. This approach has the advantage of relieving high computation, power and storage requirements of gaming from end devices. The challenge therefore shifts to streaming high quality video with low latency through fluctuating network. In this thesis, we extend the basic cloud gaming architecture in three aspects. First, we consider not only gaming as target application, but also consider much more virtual immersive applications, such as virtual classroom and virtual art gallery, etc. Second, with the development of ubiquitous wireless network for mobile devices, we specifically consider streaming the video through mobile wireless network. Third, with the growing popularity of mobile auto-stereoscopic 3D displays, we consider capture, record and stream two views (left and right views) in the virtual world so that it can render 3D views on mobile 3D displays. To conclude, we term our extended architecture as cloud mobile 3D display virtual immersive application architecture. Because 3D videos contain two views that doubles video bit rate requirement for the same quality versus 2D video, and also because mobile wireless network is much more fluctuating than wired and WiFi network, streaming high quality 3D video with low latency becomes much more challenging. In this thesis, based on prior research results that show human brain can compensate perceived video quality automatically when one of the views is of inferior quality, we propose asymmetric rendering where we tune graphics rendering quality to be asymmetric in rendering engine. We develop both user experience models and bit rate models by applying different rendering settings and conducting subjective tests. We further develop optimization algorithms which use the above two models to automatically decide the optimal rendering settings for left view and right view to ensure the best user experience given the network conditions. Experiments conducted using real 4G-LTE network profiles on commercial cloud service with different genres of applications demonstrate significant improvement in user experience when the proposed technology is applied