Machine Learning in Large-scale Genomics: Sensing, Processing, and Analysis

Advances in the field of genomics, a branch of biology concerning with the structure, function, and evolution of genomes, has led to dramatic reductions in the price of sequencing machines. As a result, torrents of genomic data is being produced every day which pose huge challenges and opportunities for engineers, scientists, and researchers in various fields. Here, we propose novel machine learning tools and algorithms to more efficiently sense, process, and analyze large-scale genomic data. To begin with, we develop a novel universal microbial diagnostics (UMD) platform to sense microbial organisms in an infectious sample, using a small number of random DNA probes that are agnostic to the target genomic DNA sequences. Our platform leverages the theory of sparse signal recovery (compressive sensing) to identify the composition of a microbial sample that potentially contains thousands of novel or mutant species. We next develop a new sensor selection algorithm that finds the subset of sensors that best recovers a sparse vector in sparse recovery problems. Our proposed algorithm, Insense, minimizes a coherence-based cost function that is adapted from classical results in sparse recovery theory and outperforms traditional selection algorithms in finding optimal DNA probes for microbial diagnostics problem. Inspired by recent progress in robust optimization, we then develop a novel hashing algorithm, dubbed RHash, that minimizes the worst-case distortion among pairs of points in a dataset using an \ell_infinity-norm minimization technique. We develop practical and efficient implementations of RHash based on the alternating direction method of multipliers (ADMM) framework and column generation that scale well to large datasets. Finally, we develop a novel machine learning algorithm using techniques in deep learning and natural language processing literature to embed DNA sequences of arbitrary length into a single low-dimensional space. Our so-called Kmer2Vec platform learns biological concepts such as drug-resistance by parsing raw DNA sequences of microbial organisms with no prior biology knowledge