Subjective and objective video quality assessment of foveated videos in virtual reality

Virtural Reality (VR) has experienced a substantial growth in popularity, due to recent advancements in consumer head-mounted displays (HMDs) and associated computing hardware technologies. In VR, the requirements of much higher resolution and smooth viewing experiences under rapid and often real-time changes in viewing direction, leads to significant challenges in compression and communication. To reduce the stresses of very high bandwidth consumption, the concept of foveated video compression is being accorded renewed interest. By exploiting the space-variant property of retinal visual acuity, foveation has the potential to substantially reduce video resolution in the visual periphery, with hardly noticeable perceptual quality degradations. Accordingly, foveated image / video quality predictors are also becoming increasingly important, as a practical way to monitor and control future foveated compression algorithms. Towards advancing the development of foveated compression / streaming algorithms, we first constructed 2D and (stereoscopic) 3D VR databases of foveated / compressed videos, called LIVE-Facebook Technologies Foveation-Compressed Virtual Reality (LIVE-FBT-FCVR) databases, and conducted a human study of perceptual quality on each database. Then we devised a no-reference (NR) foveated video quality assessment model, called FOVQA, which is based on new models of space-variant natural scene statistics (NSS) and natural video statistics (NVS). We find that FOVQA achieves state-of-the-art (SOTA) performance on the new 2D LIVE-FBT-FCVR database, as compared with other leading FVQA / VQA models. We also designed a full reference (FR) model called spatial-temporal foveated entropic differencing (STFED), which for the first time combines natural scene statistics with a contrast sensitivity function. The STFED model also achieved SOTA performance on the 2D LIVE-FBT-FCVR database in the category of FR models.Electrical and Computer Engineerin