Add to ORKGPre-trained multilingual language models show significant performance gains for zero-shot cross-lingual model transfer on a wide range of natural language understanding (NLU) tasks. Previously, for zero-shot cross-lingual evaluation, pre-trained models are only fine-tuned on English data and tested on a variety of target languages. In this paper, we do cross-lingual evaluation on various NLU tasks (sentence classification, sequence labeling, question answering) using prompt-tuning and compare it with fine-tuning. The results show that prompt tuning achieves much better cross-lingual transfer than fine-tuning across datasets, with only 0.1% to 0.3% tuned parameters. Additionally, we demonstrate through the analysis that prompt tuning can hav...
For multilingual sequence-to-sequence pretrained language models (multilingual Seq2Seq PLMs), e.g. m...
Supervised deep learning-based approaches have been applied to task-oriented dialog and have proven ...
Large-scale models for learning fixed-dimensional cross-lingual sentence representations like LASER ...
Prompt-based tuning has been proven effective for pretrained language models (PLMs). While most of t...
Transfer learning from large language models (LLMs) has emerged as a powerful technique to enable kn...
In cross-lingual language understanding, machine translation is often utilized to enhance the transf...
Cross-lingual transfer learning with large multilingual pre-trained models can be an effective appro...
Some Transformer-based models can perform cross-lingual transfer learning: those models can be train...
Cross-lingual Machine Reading Comprehension (xMRC) is a challenging task due to the lack of training...
Cross-lingual models trained on source language tasks possess the capability to directly transfer to...
Cross-lingual semantic parsing transfers parsing capability from a high-resource language (e.g., Eng...
While recent work on multilingual language models has demonstrated their capacity for cross-lingual ...
It has been shown for English that discrete and soft prompting perform strongly in fewshot learning ...
Intermediate-task training—fine-tuning a pretrained model on an intermediate task before fine-tuning...
While several benefits were realized for multilingual vision-language pretrained models, recent benc...
For multilingual sequence-to-sequence pretrained language models (multilingual Seq2Seq PLMs), e.g. m...
Supervised deep learning-based approaches have been applied to task-oriented dialog and have proven ...
Large-scale models for learning fixed-dimensional cross-lingual sentence representations like LASER ...
Prompt-based tuning has been proven effective for pretrained language models (PLMs). While most of t...
Transfer learning from large language models (LLMs) has emerged as a powerful technique to enable kn...
In cross-lingual language understanding, machine translation is often utilized to enhance the transf...
Cross-lingual transfer learning with large multilingual pre-trained models can be an effective appro...
Some Transformer-based models can perform cross-lingual transfer learning: those models can be train...
Cross-lingual Machine Reading Comprehension (xMRC) is a challenging task due to the lack of training...
Cross-lingual models trained on source language tasks possess the capability to directly transfer to...
Cross-lingual semantic parsing transfers parsing capability from a high-resource language (e.g., Eng...
While recent work on multilingual language models has demonstrated their capacity for cross-lingual ...
It has been shown for English that discrete and soft prompting perform strongly in fewshot learning ...
Intermediate-task training—fine-tuning a pretrained model on an intermediate task before fine-tuning...
While several benefits were realized for multilingual vision-language pretrained models, recent benc...
For multilingual sequence-to-sequence pretrained language models (multilingual Seq2Seq PLMs), e.g. m...
Supervised deep learning-based approaches have been applied to task-oriented dialog and have proven ...
Large-scale models for learning fixed-dimensional cross-lingual sentence representations like LASER ...