Add to ORKGWe propose PromptBERT, a novel contrastive learning method for learning better sentence representation. We firstly analyze the drawback of current sentence embedding from original BERT and find that it is mainly due to the static token embedding bias and ineffective BERT layers. Then we propose the first prompt-based sentence embeddings method and discuss two prompt representing methods and three prompt searching methods to make BERT achieve better sentence embeddings. Moreover, we propose a novel unsupervised training objective by the technology of template denoising, which substantially shortens the performance gap between the supervised and unsupervised settings. Extensive experiments show the effectiveness of our method. Compared to Sim...
Prompt-based learning has shown considerable promise in reformulating various downstream tasks as cl...
Contrastive learning has been demonstrated to be effective in enhancing pre-trained language models ...
Modern text classification models are susceptible to adversarial examples, perturbed versions of the...
Semantic representation learning for sentences is an important and well-studied problem in NLP. The ...
Though achieving impressive results on many NLP tasks, the BERT-like masked language models (MLM) en...
Recent advances in End-to-End (E2E) Spoken Language Understanding (SLU) have been primarily due to e...
Recent works have shown promising results of prompt tuning in stimulating pre-trained language model...
Zero-shot slot filling has received considerable attention to cope with the problem of limited avail...
Pre-training complex language models is essential for the success of the recent methods such as BERT...
Large pre-trained language models such as BERT have been the driving force behind recent improvement...
In recent years, pre-trained models have become dominant in most natural language processing (NLP) t...
Several prior studies have suggested that word frequency biases can cause the Bert model to learn in...
Contrastive learning-based methods, such as unsup-SimCSE, have achieved state-of-the-art (SOTA) perf...
Most adversarial attack methods that are designed to deceive a text classifier change the text class...
BERT has attained state-of-the-art performance for extractive overview tasks on the CNN/Daily-Mail d...
Prompt-based learning has shown considerable promise in reformulating various downstream tasks as cl...
Contrastive learning has been demonstrated to be effective in enhancing pre-trained language models ...
Modern text classification models are susceptible to adversarial examples, perturbed versions of the...
Semantic representation learning for sentences is an important and well-studied problem in NLP. The ...
Though achieving impressive results on many NLP tasks, the BERT-like masked language models (MLM) en...
Recent advances in End-to-End (E2E) Spoken Language Understanding (SLU) have been primarily due to e...
Recent works have shown promising results of prompt tuning in stimulating pre-trained language model...
Zero-shot slot filling has received considerable attention to cope with the problem of limited avail...
Pre-training complex language models is essential for the success of the recent methods such as BERT...
Large pre-trained language models such as BERT have been the driving force behind recent improvement...
In recent years, pre-trained models have become dominant in most natural language processing (NLP) t...
Several prior studies have suggested that word frequency biases can cause the Bert model to learn in...
Contrastive learning-based methods, such as unsup-SimCSE, have achieved state-of-the-art (SOTA) perf...
Most adversarial attack methods that are designed to deceive a text classifier change the text class...
BERT has attained state-of-the-art performance for extractive overview tasks on the CNN/Daily-Mail d...
Prompt-based learning has shown considerable promise in reformulating various downstream tasks as cl...
Contrastive learning has been demonstrated to be effective in enhancing pre-trained language models ...
Modern text classification models are susceptible to adversarial examples, perturbed versions of the...