Extracting and utilizing hidden structures in large datasets

The hidden structure within datasets --- capturing the inherent structure within the data not explicitly captured or encoded in the data format --- can often be automatically extracted and used to improve various data processing applications. Utilizing such hidden structure enables us to potentially surpass traditional algorithms that do not take this structure into account. In this thesis, we propose a general framework for algorithms that automatically extract and employ hidden structures to improve data processing performance, and discuss a set of design principles for developing such algorithms. We provide three examples to demonstrate the power of this framework in practice, showcasing how we can use hidden structures to either outperform state-of-the-art methods, or enable new applications that are previously impossible. We believe that this framework can offer new opportunities for the design of algorithms that surpass the current limit, and empower new applications in database research and many other data-centric disciplines