Combination of SSIM and JND with Content-Transition Classification for Image Quality Assessment

在影像處理演算法中，圖像品質評量(Image Quality Assessment)扮演著重要且決定性的因素。目前在此領域中，有許多先進的影像品質評量演算法，以目前較具指標性的結構相似性指標(Structural Similarity, SSIM)而言，它能經由模擬人類視覺系統(Human Visual System, HVS)在觀看影像時抽取場景中物體結構性資訊以判別圖像品質的特性，來準確地預測出人類視覺系統所感知到的圖像品質。然而，儘管結構相似性指標相較於其它方法具優越性，但對於模糊的失真圖像卻無法準確地預測圖像品質。本篇論文提出一個嶄新的圖像品質評量指標 ─JND-SSIM，此演算法是基於最小可察覺差異(Just-Noticeable Difference, JND)演算法來將圖像中像素區塊區分為平坦、邊緣及紋理區域並得到該圖像的視覺閾值模型。接著，基於各個像素區塊經過失真後像素區塊類型的轉變，再針對不同類型的變化及圖像的視覺閾值模型，對該像素區塊原先的結構相似指標值給予不同權重及比例縮減的計算來得到更準確的圖像品質評量指標。我們將提出的演算法─JND-SSIM，在目前最具公信力的兩大影像品質資料庫LIVE和TID中進行測試，實驗後的數據可證明JND-SSIM更能夠準確地符合人眼視覺系統的結果。 最後我們將提出的JND-SSIM演算法整合進視訊會議系統中，並且不論此視訊會議系統是採用軟體或硬體視訊編碼器所設計的，我們的演算法都能夠被整合於系統中。在應用裡，我們修改原先視訊會議系統中碼率控制流程，使得碼率控制模組會依據JND-SSIM結果來決定視訊壓縮碼率。然而，根據實驗結果將JND-SSIM應用於視訊會議系統中，可有效節省視訊串流碼率並且得到幾乎相同的視覺圖像品質。Image quality assessment (IQA) is a crucial feature of many image processing algorithms. The state-of-the-art IQA index, the structural similarity (SSIM) index, has been able to accurately predict image quality by assuming that the human visual system (HVS) separates structural information from nonstructural information in a scene. However, the precision of SSIM is relatively lacking when used to access blurred images. This paper proposes a novel metric of image quality assessment, the JND-SSIM, which adopts the just-noticeable di erence (JND) algorithm to di erentiate between plain, edge, and texture blocks and obtain a visibility threshold map. Based on varying block transition types between the reference and distorted image, SSIM values are assigned respective weights and scaled down by visibility threshold map. We then test our algorithm on the LIVE and TID Image Quality Database, thereby demonstrating that our improved IQA index is much closer to human opinion. In this work, we integrate our IQA tool into the video conferencing system, and regardless of video conferencing systems are designed by software or hardware encoder architecture. We modi fied the bitrate control flow of the video conferencing system and the bitrate control module will based on our IQA value to decide bitrate setting. The experimental results show signi cant bitrate savings compared with existing video conferencing system at the same subjective quality.Abstract vii 1 Introduction 1 1.1 Research Background . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.3 Thesis Organization . . . . . . . . . . . . . . . . . . . . . . . 4 2 Related Work - Structural Similarity Index 6 2.1 Structural Similarity Index Brief . . . . . . . . . . . . . . . . 6 2.2 Experimental Performance . . . . . . . . . . . . . . . . . . . 7 2.3 Limitation of SSIM . . . . . . . . . . . . . . . . . . . . . . . 8 3 Related Work - Just-noticeable Di erence 11 3.1 JND Model Brief . . . . . . . . . . . . . . . . . . . . . . . . 11 3.2 DCT-based JND estimation . . . . . . . . . . . . . . . . . . 12 3.3 Pixel-wise JND estimation . . . . . . . . . . . . . . . . . . . 15 3.4 Experimental Performance . . . . . . . . . . . . . . . . . . . 18 4 JND-SSIM: Combination of SSIM and JND 21 4.1 Block Diagram of JND-SSIM . . . . . . . . . . . . . . . . . 21 4.2 Block-transition classi cation . . . . . . . . . . . . . . . . . 23 4.3 Determining weights . . . . . . . . . . . . . . . . . . . . . . 26 4.4 Determining scale down value by NPD values . . . . . . . . 26 i 4.5 JND-SSIM . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 5 Experimental Results of the Proposed Model 30 5.1 Comparison on 4 distorted images having similar SSIM values 30 5.2 Experimental comparison on LIVE and TID . . . . . . . . . 31 5.3 Test results on LIVE and TID . . . . . . . . . . . . . . . . . 32 6 Application: Video Conferencing System with JND-SSIM 36 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 6.2 Proposal & Block digram . . . . . . . . . . . . . . . . . . . . 39 6.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . 41 7 Conclusion 46 Bibliography 4