Session management for enabling multi-party interactivity and large-scale viewing in 3D tele-immersion

Over the last few years, we have seen a surge of interest in telepresence video collaborative technologies where remote users perform virtual-reality-like interactions with each other using real-time 3D video. Though the initial focus was on video conferencing applications,recent developments in audio-video and network technologies have expanded the horizon by supporting full-body interaction for many other physical activities in the virtual environment. Despite great potential, 3D tele-immersive (3DTI) sessions still face significant challenges due to high interactivity, multi-stream dependency, multi-view dynamism and large resource demand. Quality of service (QoS) allocation in 3DTI sessions not only depends on the availability of network and processing resources, but also on the users' expectations, which are dynamic and heterogenous. In this dissertation, we revisit the design space of 3DTI session management framework. We propose a novel, comprehensive, user-experience-aware, programmable, cross-layer session control framework for 3DTI applications. Our control framework consists of a distributed monitoring oracle, a decision engine for computing user-experience-aware multi-stream dissemination topology, and a controller for allocating resources and configuring application and network layer data plane at the users. The monitoring oracle allows run-time session queries by arranging application and system layer metadata over a tree-based overlay. It resolves multi-attribute range-based performance queries in real-time without interfering application data traffic. Based on the monitored resources, performance queries, and user demands, we compute a multi-stream content dissemination topology and allocate QoS among the users. Due to the differences in interactivity requirement and scale, we consider separate content distribution solutions for immersive users (who interact with each other in the shared virtual environment in real-time) and non-immersive users (who watch the collaborative activities performed by the immersive users, but do not immerse themselves into the virtual world). The number of immersive users are small, however, the number of non-immersive users can be in the order of thousands. The immersive user prioritizes interactivity and drops delayed streams to ensure consistency across multiple geographically distributed streams. QoS allocation in this domain influences 3DTI experience of the users based on the on-going 3DTI activity. Different activities require different performance QoS profiles to ensure strong quality of experience (QoE) of the users. We model the problem of multi-stream QoS allocation as a prioritized multi-objective optimization problem, and solve it using an evolutionary approach. We show that the prioritized optimization of QoS meets expectations of the users up to 50% higher compared to the existing content distribution solutions in the 3DTI space. To further improve the interactivity in the immersive session, we introduce application and network layer synergy. It reduces multi-stream management overhead of the application nodes by splitting the application data plane responsibility, and offloading it partially to the network layer. To control the network layer components during the session run-time, we leverage support from Software Defined Networking (SDN). We show that our solution improves 3DTI interactivity and resource usage at the data plane compared to the current solutions. Furthermore, it keeps data plane robust and ensures seamless visual experience against host failures and frequent topology changes during immersive sessions. On the other hand, the non-immersive users prioritize the visual quality at the users rather than end-to-end latency. It requires to preserve multi-party multi-stream visual dependency at the display in addition to meet the usual challenges available in 2D VoD (video-on-demand) applications. Buffers are not enough as the number of streams and the size of 3D frames are both large. To solve the multi-stream content distribution and dependency preservation problem, we first design a CDN-P2P based hybrid multi-stream content distribution structure by considering users' views, stream priorities, network link bandwidth, and end-to-end delay of the streams. Second, we develop a distributed content caching technique, by organizing the non-immersive users in a logical hierarchy, based on multi-stream latencies. We show that our solution improves viewing quality and improves bandwidth utilization up to 55%, and supports a large number of concurrent non-immersive users