N 4-Fields: Neural Network Nearest Neighbor Fields for Image Transforms

We propose a new architecture for difficult image processing operations, such as natural edge detection or thin object segmentation. The architecture is based on a simple combination of convolutional neural networks with the nearest neighbor search. We focus our attention on the situations when the desired image transformation is too hard for a neural network to learn explicitly. We show that in such situations, the use of the nearest neighbor search on top of the network output allows to im-prove the results considerably and to account for the underfitting effect during the neural network training. The approach is validated on three challenging bench-marks, where the performance of the proposed architecture matches or exceeds the state-of-the-art.