Add to ORKGWhile pretrained language models (PLMs) primarily serve as general purpose text encoders that can be fine-tuned for a wide variety of downstream tasks, recent work has shown that they can also be rewired to produce high-quality word representations (i.e., static word embeddings) and yield good performance in type-level lexical tasks. While existing work primarily focused on lexical specialization of PLMs in monolingual and bilingual settings, in this work we expose massively multilingual transformers (MMTs, e.g., mBERT or XLM-R) to multilingual lexical knowledge at scale, leveraging BabelNet as the readily available rich source of multilingual and cross-lingual type-level lexical knowledge. Concretely, we leverage BabelNet's multilingual sy...
International audienceTransfer learning based on pretraining language models on a large amount of ra...
Some Transformer-based models can perform cross-lingual transfer learning: those models can be train...
Large Language Models (LLMs), trained predominantly on extensive English data, often exhibit limitat...
Pre-trained multilingual language models play an important role in cross-lingual natural language un...
Multilingual neural machine translation (MNMT) trained in multiple language pairs has attracted cons...
Multilingual language models are widely used to extend NLP systems to low-resource languages. Howeve...
Recently, it has been found that monolingual English language models can be used as knowledge bases....
In this paper, we introduce DOCmT5, a multilingual sequence-to-sequence language model pretrained wi...
Large pretrained multilingual models, trained on dozens of languages, have delivered promising resul...
For multilingual sequence-to-sequence pretrained language models (multilingual Seq2Seq PLMs), e.g. m...
Dense retrieval models have predominantly been studied for English, where models have shown great su...
Large multilingual language models typically rely on a single vocabulary shared across 100+ language...
Pretrained multilingual language models have become a common tool in transferring NLP capabilities t...
Supervised deep learning-based approaches have been applied to task-oriented dialog and have proven ...
In this paper, we introduce a massively multilingual speech corpora with fine-grained phonemic trans...
International audienceTransfer learning based on pretraining language models on a large amount of ra...
Some Transformer-based models can perform cross-lingual transfer learning: those models can be train...
Large Language Models (LLMs), trained predominantly on extensive English data, often exhibit limitat...
Pre-trained multilingual language models play an important role in cross-lingual natural language un...
Multilingual neural machine translation (MNMT) trained in multiple language pairs has attracted cons...
Multilingual language models are widely used to extend NLP systems to low-resource languages. Howeve...
Recently, it has been found that monolingual English language models can be used as knowledge bases....
In this paper, we introduce DOCmT5, a multilingual sequence-to-sequence language model pretrained wi...
Large pretrained multilingual models, trained on dozens of languages, have delivered promising resul...
For multilingual sequence-to-sequence pretrained language models (multilingual Seq2Seq PLMs), e.g. m...
Dense retrieval models have predominantly been studied for English, where models have shown great su...
Large multilingual language models typically rely on a single vocabulary shared across 100+ language...
Pretrained multilingual language models have become a common tool in transferring NLP capabilities t...
Supervised deep learning-based approaches have been applied to task-oriented dialog and have proven ...
In this paper, we introduce a massively multilingual speech corpora with fine-grained phonemic trans...
International audienceTransfer learning based on pretraining language models on a large amount of ra...
Some Transformer-based models can perform cross-lingual transfer learning: those models can be train...
Large Language Models (LLMs), trained predominantly on extensive English data, often exhibit limitat...