Smart Color Image Recognition for Steel Bridge Rust Inspection

影像處理已廣泛用於學術研究及產業，於公共工程維護檢測之應用，包括鋼橋塗漆檢測及下水道管壁檢測等。雖然相關文獻顯示，K-Means聚類法是最有效的鋼橋生鏽偵測法，但此法仍無法穩定地辨識光度不均之影像，以及輕微生鏽部分；且在過去應用影像處理的鋼橋檢測研究中，尚無有效的模型可解決照片中光度不均之問題，亦未發展出自動化的彩色辨識系統。因此，本研究以鋼橋塗漆生鏽檢測為例，處理此二問題，並以發展自動化模型為目標。為選取抵抗光度不均能力較佳之彩色座標，本研究首先從現今十四個常見的彩色座標系統中，選取相對最佳的生鏽辨識之彩色座標。經由實驗決定a*b*座標為最具抵抗光度不均能力之座標，本研究並以此座標發展以下兩個模型：Adaptive ellipse approach (AEA) 及 Box and ellipse-based neural fuzzy approach (BENFA)。第一個模型Adaptive ellipse approach (AEA)中，一張生鏽影像被分為三個區域，生鏽、背景(即塗漆顏色)及輕微生鏽到背景顏色之漸變色區域。此模型可適當處理漸變色區域，排除光度不均之影響，以達到輕微生鏽辨識的目的。透過自動偵測背景，可決定基本的背景色；由收集的生鏽照片，作者以基本橢圓形定義生鏽顏色。本模型藉由擴大基本橢圓形加強偵測輕微生鏽顏色之成效，其擴大百分比取決於生鏽顏色與塗漆顏色之關係。與K-Means聚類法之處理結果比較後，顯示此模型可更適當地辨識輕微生鏽區域。然而，當生鏽影像顏色分佈近似平行於基本橢圓形長軸時，AEA無法適當地辨識輕微生鏽顏色。有鑑於此，作者發展第二個模型Box and ellipse-based neural fuzzy approach (BENFA)以強化漸變色區域處理。本模型應用調適性網路模糊推論系統(Adaptive-network-based fuzzy inference system)描述漸變色。為達到自動化辨識之目的，此模型引用自動偵測背景、光度調整及基本橢圓形，以決定輕微生鏽和嚴重生鏽的門檻值。研究發現，相較於Fuzzy C-Means聚類法，此模型可更穩定地辨識鋼橋表面的生鏽程度。最後，為修正光度不均之生鏽照片，作者發展第三個模型BEMD-morphology approach (BMA)。此模型應用二維經驗模態分解法(bidimensional empirical mode decomposition)降低陰影之影響，並且應用影像形態學(morphology)重建反光點之顏色。結果顯示，以K-Means聚類法處理經由此模型修正後之影像的結果，比起處理未修正影像時更接近實況。Image processing has been widely utilized in scientific research and prevalently adopted in industries. Application in infrastructure condition assessment includes defect recognition on steel bridge painting and underground sewer systems. Nevertheless, there is still no robust method to overcome the non-uniform illumination problem. Although, the K-Means is recognized as one of the best rust defect recognition methods, it cannot recognize the non-uniform illuminated images and the mild rust color well. Also, there is lack of an automated color image recognition system in this field.his research starts with an investigation of 14 color spaces in order to find out a comparatively proper color configuration for non-uniformly illuminated rust image segmentation. Among the 14 color spaces, the color configuration of a*b*, which has moderate ability to filter light, is utilized to develop the proposed two models, adaptive ellipse approach (AEA) and box and ellipse-based neural fuzzy approach (BENFA).n the adaptive ellipse approach (AEA), a rust image is partitioned into three parts, background, rust, and the gradual change color from mild-rust to background. The main idea is to deal with the gradual color change properly for mild rust color extraction. The background colors can be automatically detected from a rust image. A fundamental ellipse is previously defined by the collection of rust colors. The AEA enlarges the fundamental ellipse to include part of the gradual change in color, and the enlarged size depends on the relationship between the rust color and the color of coating. The AEA is expected to deal with the boundary between background color and rust color properly. In addition, illumination adjustment is adopted in this model in order to overcome the non-uniform illumination problem. Finally, the processing results of the AEA are compared with the K-Means clusters method to show that it can recognize the mild-rust-colors.hen the color distribution is almost parallel to the major axis of the fundamental ellipse, the proposed AEA may not recognize the mild-rust-colors well. Therefore, the box and ellipse-based neural fuzzy approach (BENFA) is proposed to deal with the gradual color change from mild-rust to background. The BENFA applies the adaptive-network-based fuzzy inference system (ANFIS) to describe the gradual change colors. In order to achieve automated detection, the BENFA applies the automated detection of background, illumination adjustment, and the fundamental ellipse to determine the thresholds of serious rust and mild rust. Compared to the Fuzzy C-Means (FCM), the BENFA can stably recognize the rust intensity.he third model which is called BEMD-morphology approach (BMA) aims to adjust the color of a non-uniformly illuminated rust image. The BMA applies the bidimesional empirical mode decomposition (BEMD) to mitigate the shade/shadow effect, and morphology to substitute the highlight points by the neighboring colors. Processing a rust image with the BMA is more reliable than processing without the BMA.inally, conclusions will be drawn and recommendations for future work will be made.摘要 IIBSTRACT IVHAPTER 1 INTRODUCTION 1.1 Background 1.2 Problem Statement 2.3 Research Objectives 3.4 Organization of the Research 4HAPTER 2 LITERATURE REVIEW 6.1 Image Processing and Its Application in Infrastructure Condition Assessment 6.1.1 Underground Pipeline 6.1.2 Steel bridge Surface Inspection 7.2 Color Features 9.2.1 Color Image 9.2.2 Color Space and Color Configuration 10.3 Color Image Segmentation 19.3.1 Feature Extraction 19.3.1.1 Color Similarity 19.3.1.2 Histogram 22.3.1.3 Entropy 22.3.1.4 Principal Component Analysis 23.3.2 Threshold Segmentation 26.3.3 Clustering Segmentation 27.3.3.1 K-Means Algorithm 28.3.3.2 Fuzzy C-Means Algorithm 30.4 Artificial Intelligence 33.4.1 Fuzzy Logic Technique 33.4.1.1 Fuzzy Sets 34.4.1.2 Membership Functions 35.4.1.3 Fuzzy Logical Operations 36.4.1.4 Defuzzification 37.4.1.5 Fuzzy Inference 39.4.2 Artificial Neural Network 40.4.2.1 Artificial Neurons 40.4.2.2 Learning Algorithm 42.4.2.3 Framework 43.4.3 Adaptive Network-Based Fuzzy Inference System (ANFIS) 45.4.3.1 Framework of ANFIS 45.4.3.2 Learning rule 47.4.3.3 Sugeno-type fuzzy inference system 51.5 Hilbert-Huang Transform 52.5.1 Hilbert-Huang Transform 52.5.2 Bidimensional Empirical Mode Decomposition (BEMD) 55.6 Morphological Image Processing 59.6.1 Dilation and Erosion 60.6.2 Opening, Closing, and TopHat Transform 61HAPTER 3 COMPARISON OF COLOR CONFIGURATION USING THE K-MEANS ALGORITHM 64.1 Characteristic of Rust Images 64.1.1 Definition of a rust color 65.1.2 Why not K-Means 67.1.3 Principal Component Features 69.2 Artificial Rust Image 72.3 Evaluation the Best Color Feature for Rust Images 74.3.1 Uniform Illumination 74.3.2 Non-uniform Illumination 76.3.3 Summary 76.4 Test the Power of L*a*b* 77.5 Summary of Chapter 81HAPTER 4 ADAPTIVE ELLIPSE APPROACH (AEA) 82.1 Rust Image Preprocessing 82.1.1 Automated Detection of Background 82.1.2 Illumination Adjustment 84.1.3 Definition of rust color by the fundamental ellipse 86.2 Adaptive Ellipse Approach 87.3 Discussion and Comparison 94.3.1 Comparison of K-Means in grayscale, RGB, and a*b* 94.3.2 Comparison of K-Means with AEA 97.4 Summary of Chapter 100HAPTER 5 BOX-AND-ELLIPSE-BASED NEURO- FUZZY APPROACH (BENFA) FOR BRIDGE COATING ASSESSMENT 101.1 Preparation of Training Data 102.1.1 Input Features 102.1.1. Chrominance of the L*a*b* and its eigenvector 103.1.1.2 Chrominance of the L*a*b* and intensity L* 105.1.2 Output Designation 108.1.2.1 Output Value Setting 108.1.2.2 Designation Strategy 110.1.2.2.1 Entropy of Chrominance lower than four 111.1.2.2.2 Entropy of Chrominance higher than four 112.1.3 Illumination Adjustment of Input Image 113.1.4 Division of Training Sets 116.2 Development of Membership Functions 120.2.1 Shape of Membership Function 120.2.2 Number of Membership Function 122.3 Automation of Rust Recognition by ANFIS 124.3.1 Picking Sub-ANFIS 124.3.2 Automated Determination of Two Thresholds 125.4 Stepwise BENFA model 129.5 Applications of BENFA approach 132.6 Summary of Chapter 135HAPTER 6 Illumination Adjustment for Bridge Coating Images Using BEMD-Morphology Approach (BMA) 136.1 Shade and Shadow Elimination by Empirical Mode Decomposition 136.2 Highlight Substitution by Rust Color 142.2.1 Morphological Reconstruction for Brightness Detection 143.2.2 Seed Points of Highlight 145.2.3 Highlight Detection 146.2.4 Color Substitution 148.2.5 Application of Highlight Substitution by Rust Color 150.3 Illumination Adjustment Strategy -BMA 154.3.1 Development of the BMA 155.3.2 Stepwise the BMA 156.3.3 Evaluation of the BMA using the K-Means 158.4 Summary of Chapter 159HAPTER 7 CONCLUSIONS AND RECOMMENDATIONS 162.1 Conclusions of Research 162.2 Research Contributions 166.3 Limitations 167.4 Recommendations for Future Work 168eferences 17