As the representation capability of Pre-trained Language Models (PLMs) improve, there is growing concern that they will inherit social biases from unprocessed corpora. Most previous debiasing techniques used Counterfactual Data Augmentation (CDA) to balance the training corpus. However, CDA slightly modifies the original corpus, limiting the representation distance between different demographic groups to a narrow range. As a result, the debiasing model easily fits the differences between counterfactual pairs, which affects its debiasing performance with limited text resources. In this paper, we propose an adversarial training-inspired two-stage debiasing model using Contrastive learning with Continuous Prompt Augmentation (named CCPA) to mi...
Large pre-trained language models are successfully being used in a variety of tasks, across many lan...
Likelihood, although useful as a training loss, is a poor search objective for guiding open-ended ge...
Despite exciting progress in large-scale language generation, the expressiveness of its representati...
Pre-trained Language Models are widely used in many important real-world applications. However, rece...
Pre-trained language models reflect the inherent social biases of their training corpus. Many method...
Natural language understanding (NLU) models often rely on dataset biases rather than intended task-r...
Several prior studies have suggested that word frequency biases can cause the Bert model to learn in...
Despite their impressive performance in a wide range of NLP tasks, Large Language Models (LLMs) have...
Several works have proven that finetuning is an applicable approach for debiasing contextualized wor...
Pretrained language models can be effectively stimulated by textual prompts or demonstrations, espec...
Cheap-to-Build Very Large-Language Models (CtB-LLMs) with affordable training are emerging as the ne...
Warning: this paper contains model outputs exhibiting offensiveness and biases. Recently pre-trained...
Recent discoveries have revealed that deep neural networks might behave in a biased manner in many r...
Language Representation Models (LRMs) trained with real-world data may capture and exacerbate undesi...
When trained on large, unfiltered crawls from the Internet, language models pick up and reproduce al...
Large pre-trained language models are successfully being used in a variety of tasks, across many lan...
Likelihood, although useful as a training loss, is a poor search objective for guiding open-ended ge...
Despite exciting progress in large-scale language generation, the expressiveness of its representati...
Pre-trained Language Models are widely used in many important real-world applications. However, rece...
Pre-trained language models reflect the inherent social biases of their training corpus. Many method...
Natural language understanding (NLU) models often rely on dataset biases rather than intended task-r...
Several prior studies have suggested that word frequency biases can cause the Bert model to learn in...
Despite their impressive performance in a wide range of NLP tasks, Large Language Models (LLMs) have...
Several works have proven that finetuning is an applicable approach for debiasing contextualized wor...
Pretrained language models can be effectively stimulated by textual prompts or demonstrations, espec...
Cheap-to-Build Very Large-Language Models (CtB-LLMs) with affordable training are emerging as the ne...
Warning: this paper contains model outputs exhibiting offensiveness and biases. Recently pre-trained...
Recent discoveries have revealed that deep neural networks might behave in a biased manner in many r...
Language Representation Models (LRMs) trained with real-world data may capture and exacerbate undesi...
When trained on large, unfiltered crawls from the Internet, language models pick up and reproduce al...
Large pre-trained language models are successfully being used in a variety of tasks, across many lan...
Likelihood, although useful as a training loss, is a poor search objective for guiding open-ended ge...
Despite exciting progress in large-scale language generation, the expressiveness of its representati...