A Framework for example-based Synthesis of Materials for Physically Based Rendering

In computer graphics, textures are used to create detail along geometric surfaces. They are less computationally expensive than geometry, but this efficiency is traded for greater memory demands, especially with large output resolutions. Research has shown, that textures can be synthesized from low-resolution exemplars, reducing overall runtime memory cost and enabling applications, like remixing existing textures to create new, visually similar representations. In many modern applications, textures are not limited to simple images, but rather represent geometric detail in different ways, that describe how lights interacts at a certain point on a surface. Physically Based Rendering (PBR) is a technique, that employs complex lighting models to create effects like self-shadowing, realistic reflections or subsurface scattering. A set of multiple textures is used to describe what is called a material. In this thesis, example-based texture synthesis is extented to physical lighting models to create a physically based material synthesizer. It introduces a framework that is capable of utilizing multiple texture maps to synthesize new representations from existing material exemplars. The framework is then tested with multiple exemplars from different texture categories, to prospect synthesis performance in terms of quality and computation time. The synthesizer works in uv space, enabling to re-use the same exemplar material at runtime with different uv maps, reducing memory cost, whilst increasing visual varienty and minimizing repetition artifacts. The thesis shows, that this can be done effectively, without introducing inconsitencies like seams or discontiuities under dynamic lighting scenarios.:1. Context and Motivation 2. Introduction 2.1. Terminology: What is a Texture? 2.1.1. Classifying Textures 2.1.2. Characteristics and Appearance 2.1.3. Advanced Analysis 2.2. Texture Representation 2.2.1. Is there a theoretical Limit for Texture Resolution? 2.3. Texture Authoring 2.3.1. Texture Generation from Photographs 2.3.2. Computer-Aided Texture Generation 2.4. Introduction to Physically Based Rendering 2.4.1. Empirical Shading and Lighting Models 2.4.2. The Bi-Directional Reflectance Distribution Function (BRDF) 2.4.3. Typical Texture Representations for Physically Based Models 3. A brief History of Texture Synthesis 3.1. Algorithm Categories and their Developments 3.1.1. Pixel-based Texture Synthesis 3.1.2. Patch-based Texture Synthesis 3.1.3. Texture Optimization 3.1.4. Neural Network Texture Synthesis 3.2. The Purpose of example-based Texture Synthesis Algorithms 4. Framework Design 4.1. Dividing Synthesis into subsequent Stages 4.2. Analysis Stage 4.2.1. Search Space 4.2.2. Guidance Channel Extraction 4.3. Synthesis Stage 4.3.1. Synthesis by Neighborhood Matching 4.3.2. Validation 5. Implementation 5.1. Modules and Components 5.2. Image Processing 5.2.1. Image Representation 5.2.2. Filters and Guidance Channel Extraction 5.2.3. Search Space and Descriptors 5.2.4. Neighborhood Search 5.3. Implementing Synthesizers 5.3.1. Unified Synthesis Interface 5.3.2. Appearance Space Synthesis: A Hierarchical, Parallel, Per-Pixel Synthesizer 5.3.3. (Near-) Regular Texture Synthesis 5.3.4. Extented Appearance Space: A Physical Material Synthesizer 5.4. Persistence 5.4.1. Codecs 5.4.2. Assets 5.5. Command Line Sandbox 5.5.1. Providing Texture Images and Material Dictionaries 6. Experiments and Results 6.1. Test Setup 6.1.1. Metrics 6.1.2. Result Visualization 6.1.3. Limitations and Conventions 6.2. Experiment 1: Analysis Stage Performance 6.2.1. Influence of Exemplar Resolution 6.2.2. Influence of Exemplar Maps 6.3. Experiment 2: Synthesis Performance 6.3.1. Influence of Exemplar Resolution 6.3.2. Influence of Exemplar Maps 6.3.3. Influence of Sample Resolution 6.4. Experiment 3: Synthesis Quality 6.4.1. Influence of Per-Level Jitter 6.4.2. Influence of Exemplar Maps and Map Weights 7. Discussion and Outlook 7.1. Contributions 7.2. Further Improvements and Research 7.2.1. Performance Improvements 7.2.2. Quality Improvements 7.2.3. Methology 7.2.4. Further Problem Field