Feature-Based Texture Synthesis and Hierarchical Tensor Approximation

Texture synthesis has always been an interesting research topic in Graphics. The popular neighborhood-based algorithms have two common stages: search for most similar neighborhoods in the sample texture; merge a local neighborhood into the (partially) synthesized output texture. When the first stage cannot find good neighbors, the second stage may generate seams. We propose to extract a feature map from an example texture and synthesize a feature map. We develop novel algorithms to perform feature matching and alignment. This approach can significantly reduce feature discontinuities and related artifacts. For 3D surfaces, things get more complicated as continuity constraints in neighborhood-based texture synthesis seriously restrict the variability of synthesized textures. We propose to relax such restrictions and decompose synthesis into two stages: feature map synthesis and Laplacian texture reconstruction. Experiments indicate that this relaxation can produce desirable results for regular texture synthesis as well as texture mixing from multiple sources. Visual data approximation is another important issue in Graphics that also facilitates texture synthesis. We propose hierarchical tensor approximation to expose multi-scale components and spatially inhomogeneous structures in visual datasets. The blocks on each level in the hierarchy are pruned and approximated as a tensor ensemble, and the residual tensors are subdivided to form the next level in the hierarchy. Experiments prove that the hierarchical multilinear models can achieve higher compression ratios and quality on high-dimensional visual data than wavelet (packet) transforms and single-level tensor approximation. Unfortunately, multilinear models suffer basis overhead for low-dimensional data. We propose to apply the models to wavelet domain to reduce overhead. After hierarchical wavelet (packet) transform, high-frequency coefficients are subdivided into small blocks most of which have small energy and get pruned. The blocks are usually correlated especially when properly classified. Different channels and sub-bands may exhibit strong redundancy as well. We reorganize the subdivided blocks into small tensors, classify the unpruned ones and approximate each cluster as a tensor ensemble. Experiments on images and medical volume data indicate that this approach achieves better approximation quality than wavelet (packet) transforms and hybrid linear models