A browser based system for remote evaluation of subjective quality of videos

The evaluation of the quality of videos is posing a significant problem in research on video processing, including compression. Different algorithms and their modifications aim to improve the quality of the video, while keeping the necessary bitrate as low as possible, and every time, the performance of the method must be evaluated. The bitrate can be measured readily, while for quality there are different metrics and, therefore – different results. The ultimate verdict always has to be the subjective opinion about the quality that is expressed by the viewers. For now, the only reliable way to measure the quality of the video is to perform a subjective test with a group of viewers. Subjective tests are difficult to perform – not only one has to gather all viewers in one place, but also the evaluation schemes may be difficult to follow. In the paper we present an implementation of a system that greatly simplifies the process of performing the subjective quality assessment of videos. We discuss strengths and weaknesses of the system when compared to the commonly used procedures. Finally, we also provide the results of a quality assessment for the HEVC video encoder. Presented data proves that the idea of remote quality evaluation together with the usage of the two-level grade is a valid one and provides reliable results