Bridging Theory and Algorithms for Open-Set and Heterogeneous Domain Adaptations

University of Technology Sydney. Faculty of Engineering and Information Technology.The availability of massive labeled datasets results in the successes of supervised learning in many applications such as object detection, speech recognition and natural language processing. However, the curse of domain mismatch arises if the test samples (target samples) and training samples (source samples) are from different domains. To overcome the mismatch between domains, researchers have proposed transfer learning, which aims to leverage knowledge from source samples with abundant labels to help train a classifier for target samples with insufficient labels. Although many algorithms have been developed to solve transfer learning problem, most algorithms depend on additional assumptions to ensure their effectiveness. Many assumptions are difficult to be satisfied in real-world application. For example, the label sets of source and target samples are assumed to be same, however, in unsupervised setting, it is impossible to check this assumption. Those unrealistic assumptions limit the application of transfer learning. This thesis mainly studies two important problems faced by many existing transfer learning algorithms: 1) how to transfer knowledge in the unsupervised setting, when the target label set is larger than the source label set; and 2) how to transfer knowledge, when the source and target samples are from different feature spaces. Before solving problems 1) and 2), we first consider transfer learning which assumes that the feature spaces and label sets are both same. The setting is also called homogeneous domain adaptation. By studying the homogeneous domain adaptation, we aim to understand how the source knowledge can be used to help improve the target's performance under the simplest setting (i.e., homogeneous setting). To solve problem 1), this thesis considers a challenging task: unsupervised open-set domain adaptation (UOSDA). First, to understand why source samples can be used to help target samples achieve good performance in the open-set setting, we develop Probably approximately correct (PAC) learning theory for OSDA. Then, based on our theory, a kernel-based algorithm is proposed to solve problem 1). As an application of our OSDA theory, we also establish a theoretical foundation for open-set learning (OSL), an important sub-field in machine learning. To solve problem 2), this thesis studies semi-supervised heterogeneous domain adaptation (SsHeDA) problem. Motivated by the compatibility condition in semi-supervised PAC theory, we explain the SsHeDA problem by proving its generalization error–why labeled source samples and unlabeled target samples help to reduce the target error. Guided by our theory, we devise a kernel-based algorithm