Deep neural open information extraction with background knowledge

Natural language text, which exists in unstructured format, has a vast amount of knowledge about the world we live in. With the ever-increasing volume of natural language literature, it has become an incredibly time consuming effort to analyse text and extract important knowledge buried within it. This has resulted in the emergence of information extraction (IE) and natural language processing (NLP) methodologies and tools. IE focuses on the automatic extraction of structured semantic information from text. Extraction and additional analysis of these clear concepts and relationships aid in the discovery of various insights contained in text. This dissertation focuses on open information extraction (OIE), a new type of IE. Unlike traditional IE, OIE is not confined to a predetermined set of domain-specific relations and is expected to extract all relations found in natural language text. Several neural OIE algorithms have been proposed that approach OIE as either a sequence tagging or a sequence generating problem. Sequence tagging approaches identify each token in the input text as belonging to the subject, predicate, or object, whereas sequence generation approaches produce tuples one word at a time given the input text. The proposed methodologies have some limitations, which inspire this research. First, due to unequal label frequencies in OIE datasets, sequence tagging techniques tend to put too much emphasis on labels that appear at higher frequencies. Second, sequence generation systems are prone to not only producing the same fact several times, but also producing repeating tokens in facts. Third, while sequence generation systems use words from a vocabulary while constructing implicit facts, they lack features that explicitly encourage them to either use terms from the vocabulary or input text. Fourth, techniques that incorporate syntactic information by employing part-of-speech (PoS) and dependency tags in addition to the actual input text make insufficient use of the vast syntactic information, particularly that reflected in the structure of dependency trees. This dissertation aims to address the aforementioned drawbacks of earlier OIE approaches. In this research, I only examine neural OIE approaches because they have outperformed prior rule-based systems and address the issue of error propagation in rule-based systems. I give novel methodologies to tackle the limits of earlier approaches while leveraging state-of-the-art (SOTA) deep learning methods. Furthermore, I study whether incorporating factual knowledge from knowledge graphs (KGs) into neural OIE models can improve the performance of OIE approaches. First, I present three innovative training procedures for sequence labelling OIE approaches to remove model biases caused by uneven tag frequencies in OIE datasets on models. Second, I avoid creating redundant tokens in facts and increase the models’ capacity to create implicit facts by providing methods for explicitly guiding the models to use terms from either the vocabulary or input text. This strategy drastically reduces the number of recurring tokens in fact. Furthermore, when this strategy is used, the models repeat fewer tokens from the original phrase and introduce more tokens from the vocabulary, which implies a better ability to generate implicit facts. Third, I suggest a method for maximising the syntactic information provided by dependency tree topologies. Using the structure of dependency trees, I compute syntactically rich vector representations of input text tokens. Fourth, I present a knowledge-enhanced OIE framework for both sequence tagging and sequence generation OIE techniques, which builds on recent achievements in embedding knowledge in pre-trained language models (PLMs). After thorough testing, I confirm that the knowledge-enhanced OIE framework improves the performance of OIE models. Finally, I provide a unique discriminative strategy for neural OIE model training