Depth Estimation and Focus Recovery

在本篇論文中，我們探討如何由影像得到物體的景深以及如何回復失焦的影像。我們介紹幾何光學以及傅氏光學建構在平行入射光源上的基礎理論，並進而引出文獻以及我們所提出的景深求解的方法。乎所有的景深回復研究都是簡單的建立在相機的聚焦與散焦之上。在這些方法中，它們通常都會陷入景深不連續處的麻煩。然而，也有少數的方法可以解決這個問題。眼視覺和單眼視覺是兩個主要訓練影像去估計景深的作法。而雙眼視覺的基本概念來自於人眼的立體視覺。當在同一基準線上由兩邊盯著看的時候，我們才能感覺到此物體的立體感。中，於聚焦求得景深(DFF)與由散焦求得景深(DFD)則特別是單眼視覺的方法。FF是被用來估計更準確的景深值。醫學影像會有這種需求，例如內視鏡所得到之影像的診斷。DFF利用某一特定像素不同“聚焦程度”，以高斯函數進行內插法逼近出一個精確的景深值。為了達到這個方法的需求，我們需要一序列影像，此序列影像是根據不同的物體與相機間的距離而被拍攝而成。至於DFD，它是建構於求解具有不同相機參數的成像模型，參數包含焦距、光圈直徑大小以及鏡頭至感光元件間的距離。為影像模糊的呈現不僅僅是單獨點的特性，而是整個平面上的衰減，所有這些方法都需要定義出一個空間範圍(工作窗)，在這範圍裡面的模糊程度是相同的。們提出的方法將在最後兩章介紹到。我們建議可以應用線性基準轉換(LCT)以估計物體的景深值並且也可回復失焦的影像。CT的特性是它可以利用四個參數來模擬很多光學效應。我們使用LCT來推導以及了解在特定的輸入訊號下其輸出訊號的解，輸入訊號例如高斯函數和步階函數(邊緣)。關鍵點在於我們可以由此資訊得知散焦訊號與景深間的關係為何。在失焦回復的研究，我們也嘗試從失焦的影像利用LCT模擬出聚焦影像。理論上，透過實現出一個具有微小光圈孔徑的光學環境，可以造就等效的聚焦影像。然而，以LCT來做模擬是一個困難的工作。我們也會在這篇論文中討論到一些模擬問題的細節。In this thesis, we discuss how to get depth values from image and how to recover a defocused image. We introduce the fundamentals of geometric optics and Fourier optics in parallel incident light hence it leads to the training methods on depth cues of docu-ments and ours.ost of depth recovery methods are simply based on camera focus and defocus. Among those approaches, they usually fall in a depth discontinuity problem. However, there are some methods which can solve such problems.inocular vision and monocular vision methods are two main directions to train images for estimating depths. One basic idea of binocular method comes from the stereo vision of human eyes. While there is two-side gazing on the same baseline we can sense that stereo from objects. mong the monocular methods, depth from focus (DFF) and depth from defocus (DFD) are the skills specifically.FF is used to estimate a more accurate depth value of the object. It could be im-plemented on medical images, for example, the diagnosis of the image from a laparo-scope. DFF uses Gaussian interpolation of different “degree of focus” on a particular pixel to approach an accurate depth value. To do this, we need a sequence of images and it is taken with different distances between the object and camera. As to the DFD, it is based on solving imaging model referring to different camera settings which include focal length, aperture diameter and distance between shoot to sensor.ecause image blurring is performed as not only a point property of image but a spatial degrading, all of those approaches require to define a spatial window (working window) where the blurring degree inside is same.ur proposed method will be introduced in the last two chapters. We suggest ap-plying the linear canonical transform (LCT) to estimate the depth value of the object and also recover the defocused image.he LCT has the property that it can simulate many optical effects using its four parameters. We use the LCT to derive and realize the output signal with the specific in-put signals, like Gaussian function and step function (edge). The key point is that we can get the relation between defocused input signals and depth values.hile in focus recovery, we also use LCT to simulate focus images from defo-cused one. Theoretically, through estimating an optical environment with a tiny aperture diameter can make effective focus images. However, the simulation on the LCT is a tough work. We will also discuss the simulation problems in detail in this thesis.CHAPTER 1 INTRODUCTION 1.1 OVERVIEW ON DEPTH ESTIMATION 1.2 OVERVIEW ON FOCUS RECOVERY 4.3 ARRANGEMENT OF CONTENT 5HAPTER 2 PLELIMINARY WORKS 7.1 ABERRATIONS OF CAMERA 7.2 STRUCTURE OF ZOOM LENS 12.3 BLURRING MODEL AND THE GEOMETRIC OPTICS 14.4 INTRODUCTION TO FOURIER OPTICS 16HAPTER 3 OPTICS ANALYZED BY THE LINEAR CANONICAL TRANSFORM (LCT) 23.1 INTRODUCTION TO THE LCT 23.2 PROPERTIES OF THE LCT 25.3 OPTICAL EFFECTS REFERRED TO THE LCT 28.4 SIMULATION ON IMAGES BY LCTS 33.5 OPTICAL EFFECTS BY FINITE INSTRUMENTS WITH LCTS 38HAPTER 4 BINOCULAR VISION SYSTEM 43.1 PROCEDURES 43.2 TRIANGULAR GEOMETRY 45HAPTER 5 MONOCULAR VISION SYSTEM-DEPTH FROM FOCUS 49.1 MEASUREMENT OPERATOR-SUM OF LAPLACIAN 49.2 GAUSSIAN INTERPOLATION 51HAPTER 6 MONOCULAR VISION SYSTEM-DEPTH FROM DEFOCUS 53.1 TYPICAL METHODS 53.2 FILTERING IN THE WAVELET DOMAIN 57.3 INTRODUCTION TO HEAT DIFFUSION EQUATION 59.4 SPACE-VARYING VIA RELATIVE BLURRING 60.5 WORKING ON THE FORWARD DIFFUSION 62.6 GRADIENT FLOW ALGORITHM 64.7 RELATION BETWEEN THE DFD AND BINOCULAR METHODS 65.8 DEPTH ESTIMATION FROM A BLURRED IMAGE 68HAPTER 7 OTHER POSSIBLE METHODS FOR DEPTH ESTIMATION 71.1 COAXIAL PROJECTOR-CAMERA SYSTEM 71.2 DEPTH FROM PROJECTION DEFOCUS 72.3 RESTORATION FROM CAMERA MOTION BLUR 73HAPTER 8 PROPOSED BLURRING MODEL BASED ON THE LCTS 75.1 GAUSSIAN FUNCTION BLURRING MODEL BASED ON LCTS 75.2 PROPOSED DFD METHOD WITH LCT BLURRING MODELS ON THE GAUSSIAN FUNCTION 77.3 STEP FUNCTION BLURRING MODEL BASED ON LCTS 79.4 PROPOSED DFD METHOD WITH LCT BLURRING MODELS ON THE STEP FUNCTION 84HAPTER 9 FOCUS RECOVERY: THE ALTERNATIVE METHOD AND THE COMMON METHOD 85.1 DEPTH OF FIELD RELATING TO PERCEPTUAL FOCUS 85.2 PROBLEMS ON SIMULATIONS 89.3 SIMULATION ON PATTERNS 91.4 THE OTHER METHOD: RECONSTRUCTION OF TWO LEVEL FOCUSED SCENES 92HAPTER 10 CONCLUSIONS AND FUTURE WORK 95EFERENCES 9