Supervised machine learning models and their evaluation strongly depends on the quality of the underlying dataset. When we search for a relevant piece of information it may appear anywhere in a given passage. However, we observe a bias in the position of the correct answer in the text in two popular Question Answering datasets used for passage re-ranking. The excessive favoring of earlier positions inside passages is an unwanted artefact. This leads to three common Transformer-based re-ranking models to ignore relevant parts in unseen passages. More concerningly, as the evaluation set is taken from the same biased distribution, the models overfitting to that bias overestimate their true effectiveness. In this work we analyze position bias o...
The Chinese academy of sciences Information Retrieval team (CIR) has participated in the NTCIR-17 UL...
Neural ranking methods based on large transformer models have recently gained significant attention ...
The availability of massive data and computing power allowing for effective data driven neural appro...
In recent years, large pre-trained transformers have led to substantial gains in performance over tr...
Machine Reading Comprehension (MRC) models tend to take advantage of spurious correlations (also kno...
In the field of information retrieval, Passage related to Query are usually easy to get, and the pas...
Algorithmic bias presents a difficult challenge within Information Retrieval. Long has it been known...
Many recent neural models have shown remarkable empirical results in Machine Reading Comprehension, ...
Thesis (Ph.D.)--University of Washington, 2020Modern machine learning algorithms have been able to a...
Algorithmic bias presents a dificult challenge within Information Retrieval. Long has it been known ...
The emergence of BERT in 2018 has brought a huge boon to retrieval effectiveness in many tasks acros...
Position bias is a critical problem in information retrieval when dealing with implicit yet biased u...
People frequently interact with information retrieval (IR) systems, however, IR models exhibit biase...
Reordering is one of the most important factors affecting the quality of the output in statistical m...
One challenge with neural ranking is the need for a large amount of manually-labeled relevance judgm...
The Chinese academy of sciences Information Retrieval team (CIR) has participated in the NTCIR-17 UL...
Neural ranking methods based on large transformer models have recently gained significant attention ...
The availability of massive data and computing power allowing for effective data driven neural appro...
In recent years, large pre-trained transformers have led to substantial gains in performance over tr...
Machine Reading Comprehension (MRC) models tend to take advantage of spurious correlations (also kno...
In the field of information retrieval, Passage related to Query are usually easy to get, and the pas...
Algorithmic bias presents a difficult challenge within Information Retrieval. Long has it been known...
Many recent neural models have shown remarkable empirical results in Machine Reading Comprehension, ...
Thesis (Ph.D.)--University of Washington, 2020Modern machine learning algorithms have been able to a...
Algorithmic bias presents a dificult challenge within Information Retrieval. Long has it been known ...
The emergence of BERT in 2018 has brought a huge boon to retrieval effectiveness in many tasks acros...
Position bias is a critical problem in information retrieval when dealing with implicit yet biased u...
People frequently interact with information retrieval (IR) systems, however, IR models exhibit biase...
Reordering is one of the most important factors affecting the quality of the output in statistical m...
One challenge with neural ranking is the need for a large amount of manually-labeled relevance judgm...
The Chinese academy of sciences Information Retrieval team (CIR) has participated in the NTCIR-17 UL...
Neural ranking methods based on large transformer models have recently gained significant attention ...
The availability of massive data and computing power allowing for effective data driven neural appro...