An Algorithm and Hardware Design for Fourier Slice Theorem Based Digital Refocusing

在一般光學相機系統中，相機之感光器為一個二維平面，在捕捉現實世界之四維光場資料時，所擷取到之二維資訊為整個四維空間場域投影至二維感光器平面之結果。亦即，一般相機之感光器所能記錄之資訊僅為空間光場之一部分，其餘維度之資訊則無法被記錄下來。 在這篇論文中，我們設計了由Kodak LVT技術印製而成的光學遮罩，並將其放置在數位單眼相機之感光器前。在此情況下由於二維感光器接收到之光線強度為原始光場與遮罩函式之乘積，根據旋積理論(convolution theorem)，空間域中兩函式之乘積在頻率域為兩者之旋積，因此放置光罩後，感光器所擷取到之資訊在頻率域為光場資料與遮罩函式之旋積。若遮罩之圖形函式在頻率域為一系列之脈衝，則與光場資料旋積之結果將得到一系列複製之光場資料。利用這些複製之資料，我們可以得到投影於感光器平面以外之資訊，藉以還原空間中之四維光場。利用上述系統所得到之資料，根據傅立葉切片理(Fourier slice theorem)之空間域投影(projection)與頻率域切片(slice)之等價關係，我們可以將光場投影至不同感光器平面之效果以頻率域之切片來模擬，藉以合成不同之對焦距離之影像，其結果等同於傳統相機中之鏡片組變焦功能，並可依頻率域中之切面的不同來模擬不同的焦距效果。 最後，我們亦提出了數位變焦演算法之硬體架構，以提升演算法之處理速度。以TSMC13製程估計，晶片尺寸為17.58 mm2，核心尺寸為12.53 mm2，運作頻率設計為40 MHz，功率消耗為862.6 mW。In general optimal camera systems, the sensor array of a camera is on a two-dimensional plane. When a 2D sensor array is used to capture 4D light field data in the world, the data we collect is the two dimensional projection of the four dimensional light filed data. As the result, the information we get is only part of the whole 4D data, other information is lost thus can’t be recorded. In this thesis, an optical mask printed by the technique, Kodak LVT, is designed and placed in front of the sensors in a DSLR camera. In such case, the light intensities received by the two dimensional sensor array is the product of the light field data and the function of the mask. According to the convolution theorem, the data captured by camera sensors are the convolution of the light field data and the mask function in the frequency domain. If the function of the mask is a series of impulses in the frequency domain, the result of convolution is a series of copies of the light field data. By rearranging the frequency domain data, we reconstruct the information outside the projection plane of the 2D sensors, and recover all the 4D light field data in space. According to Fourier slice theorem, the operation of projection in the spatial domain is equivalent to taking a slice in the frequency domain. As the result, we can have images focused at different depths by taking different slices in the frequency domain. The approach is equivalent to having different lens settings in traditional camera systems. Finally, for future real-time applications, we also design a hardware processor using the algorithm. The chip is implemented with TSMC 130 nm technology. The chip and core sizes are 17.58 mm2 and 12.53 mm2. Power consumption is 862.6 mW when the chip operates at 40 MHz