huggingface/datasets: 2.2.0

Dataset Changes New: ImageNet by @apsdehal in https://github.com/huggingface/datasets/pull/4178 Manual download only for now New: Google Conceptual Captions by @abhishekkrthakur in https://github.com/huggingface/datasets/pull/1459 New: Conceptual 12M by @thomasw21 in https://github.com/huggingface/datasets/pull/4162 New: Visual Genome by @thomasw21 in https://github.com/huggingface/datasets/pull/4161 New: RVL-CDIP by @dnaveenr in https://github.com/huggingface/datasets/pull/4050 New: Text-based NP Enrichment (TNE) by @yanaiela in https://github.com/huggingface/datasets/pull/4153 New: TextVQA by @apsdehal in https://github.com/huggingface/datasets/pull/3967 New: ETT time series dataset by @kashif in https://github.com/huggingface/datasets/pull/4213 Update: assin2 - update metadata by @lhoestq in https://github.com/huggingface/datasets/pull/4172 Update: Librispeech - Add 'all' config by @patrickvonplaten in https://github.com/huggingface/datasets/pull/4184 Update: XGLUE - Support streaming dataset by @albertvillanova in https://github.com/huggingface/datasets/pull/4249 Update: crd3 - group all the turns in one example by @shanyas10 in https://github.com/huggingface/datasets/pull/4240 Update: pubmed_qa - Remove google drive URL by @lhoestq in https://github.com/huggingface/datasets/pull/4255 Update: SAMSum - Replace data URL dataset and support streaming by @albertvillanova in https://github.com/huggingface/datasets/pull/4254 Update: SAMSum - Replace data URL dataset within the same repository by @albertvillanova in https://github.com/huggingface/datasets/pull/4267 Update: big_patent - Replace data URL in dataset and support streaming by @albertvillanova in https://github.com/huggingface/datasets/pull/4236 Update: openbookqa - Add missing features for additional config by @albertvillanova in https://github.com/huggingface/datasets/pull/4278 Update: commonsense_qa - Add missing features by @albertvillanova in https://github.com/huggingface/datasets/pull/4280 Fix: Common Voice - Make sure bytes are correctly deleted if path exists by @patrickvonplaten in https://github.com/huggingface/datasets/pull/4212 Fix: openbookqa - fix bug in choices labels by @manandey in https://github.com/huggingface/datasets/pull/4259 Fix: openbookqa - fix style in openbookqa dataset by @albertvillanova in https://github.com/huggingface/datasets/pull/4270 Dataset Features Add support for metadata files to imagefolder by @mariosasko in https://github.com/huggingface/datasets/pull/4069 load a folder of images and metadata stored in metadata.jsonl, more info in the documentation on how to load an image dataset Infer splits when loading datasets without script by @polinaeterna in https://github.com/huggingface/datasets/pull/4144 splits are inferred from the directory and file names, see more info in the documentation on how to structure your repository Add drop_last_batch to IterableDataset.map by @mariosasko in https://github.com/huggingface/datasets/pull/4215 Load dataset with TSV files by @albertvillanova in https://github.com/huggingface/datasets/pull/4246 Dataset Cards Adding license information for Openbookcorpus by @meg-huggingface in https://github.com/huggingface/datasets/pull/3525 Make code for image downloading from image urls cacheable by @mariosasko in https://github.com/huggingface/datasets/pull/4218 Fix description links in dataset cards by @albertvillanova in https://github.com/huggingface/datasets/pull/4222 Add YAML tags to Dataset Card rotten tomatoes by @mo6zes in https://github.com/huggingface/datasets/pull/4262 Remove a copy-paste sentence in dataset cards by @albertvillanova in https://github.com/huggingface/datasets/pull/4281 Update LexGLUE README.md by @iliaschalkidis in https://github.com/huggingface/datasets/pull/4285 leadboard info added for TNE by @yanaiela in https://github.com/huggingface/datasets/pull/4273 Autoeval config by @nrajani in https://github.com/huggingface/datasets/pull/4234 Add Lahnda language tag by @mariosasko in https://github.com/huggingface/datasets/pull/4286 Add license and point of contact to big_patent dataset by @albertvillanova in https://github.com/huggingface/datasets/pull/4269 Add HF Speech Bench to Librispeech Dataset Card by @sanchit-gandhi in https://github.com/huggingface/datasets/pull/4266 Metrics Changes Perplexity Speedup by @emibaylor in https://github.com/huggingface/datasets/pull/4108 Add AUC ROC Metric by @emibaylor in https://github.com/huggingface/datasets/pull/4158 Small fixes in ROC AUC docs by @wschella in https://github.com/huggingface/datasets/pull/4239 Fix/start token mask issue and update documentation by @TristanThrush in https://github.com/huggingface/datasets/pull/4258 Add pearsonr mc, update functionality to match the original docs by @emibaylor in https://github.com/huggingface/datasets/pull/4226 Metric Cards Metric card for the XTREME-S dataset by @sashavor in https://github.com/huggingface/datasets/pull/4251 Creating metric card for MAE by @sashavor in https://github.com/huggingface/datasets/pull/4252 Create metric cards for mean IOU by @sashavor in https://github.com/huggingface/datasets/pull/4253 Create metric card for Mahalanobis Distance by @sashavor in https://github.com/huggingface/datasets/pull/4257 Create metric card for MSE by @sashavor in https://github.com/huggingface/datasets/pull/4256 Fix exact match by @emibaylor in https://github.com/huggingface/datasets/pull/4166 Fix google bleu typos, examples by @emibaylor in https://github.com/huggingface/datasets/pull/4165 Add f1 metric card, update docstring in py file by @emibaylor in https://github.com/huggingface/datasets/pull/4227 Add Recall Metric Card by @emibaylor in https://github.com/huggingface/datasets/pull/4204 Matthews Correlation Metric Card by @emibaylor in https://github.com/huggingface/datasets/pull/4110 Add Precision Metric Card by @emibaylor in https://github.com/huggingface/datasets/pull/4203 Add Accuracy Metric Card by @emibaylor in https://github.com/huggingface/datasets/pull/4223 Add Spearmanr Metric Card by @emibaylor in https://github.com/huggingface/datasets/pull/4109 Metric card template by @emibaylor in https://github.com/huggingface/datasets/pull/3915 Documentation Document save_to_disk and push_to_hub on images and audio files by @lhoestq in https://github.com/huggingface/datasets/pull/4193 Add to docs how to load from local script by @albertvillanova in https://github.com/huggingface/datasets/pull/4200 Add code examples to API docs by @stevhliu in https://github.com/huggingface/datasets/pull/4168 Add code examples for DatasetDict by @stevhliu in https://github.com/huggingface/datasets/pull/4245 Add API code examples for IterableDataset by @stevhliu in https://github.com/huggingface/datasets/pull/4274 Add packaged builder configs to the documentation by @lhoestq in https://github.com/huggingface/datasets/pull/4307 [Imagefolder] Docs + Don't infer labels from file names when there are metadata + Error messages when metadata and images aren't linked correctly by @lhoestq in https://github.com/huggingface/datasets/pull/4311 General improvements and bug fixes Generate tasks.json taxonomy from huggingface_hub by @julien-c in https://github.com/huggingface/datasets/pull/4154 Fix when map function modifies input in-place by @thomasw21 in https://github.com/huggingface/datasets/pull/4174 Support streaming cnn_dailymail dataset by @albertvillanova in https://github.com/huggingface/datasets/pull/4188 Don't duplicate data when encoding audio or image by @lhoestq in https://github.com/huggingface/datasets/pull/4187 Fix outdated docstring about default dataset config by @lhoestq in https://github.com/huggingface/datasets/pull/4186 Deprecate shard_size in push_to_hub in favor of max_shard_size by @mariosasko in https://github.com/huggingface/datasets/pull/4190 Fix some type annotation in doc by @thomasw21 in https://github.com/huggingface/datasets/pull/4202 Update GH template for dataset viewer issues by @albertvillanova in https://github.com/huggingface/datasets/pull/4201 Update auth when mirroring datasets on the hub by @lhoestq in https://github.com/huggingface/datasets/pull/4242 Rename imagenet2012 -> imagenet-1k by @lhoestq in https://github.com/huggingface/datasets/pull/4263 Skip checksum computation in Imagefolder by default by @mariosasko in https://github.com/huggingface/datasets/pull/4214 Fix convert_file_size_to_int for kilobits and megabits by @mariosasko in https://github.com/huggingface/datasets/pull/4205 Fix typo in logging docs by @stevhliu in https://github.com/huggingface/datasets/pull/4272 Bump PyArrow Version to 6 by @dnaveenr in https://github.com/huggingface/datasets/pull/4250 task id update by @nrajani in https://github.com/huggingface/datasets/pull/4244 Avoid recursion error in map if example is returned as dict value by @mariosasko in https://github.com/huggingface/datasets/pull/4216 Update minimal PyArrow version warning by @mariosasko in https://github.com/huggingface/datasets/pull/4279 [Minor edit] Fix typo in class name by @cakiki in https://github.com/huggingface/datasets/pull/4207 Stream private zipped images by @lhoestq in https://github.com/huggingface/datasets/pull/4173 Fix filesystem docstring by @stevhliu in https://github.com/huggingface/datasets/pull/4283 Document how to use FAISS index for special operations by @albertvillanova in https://github.com/huggingface/datasets/pull/4189 Contributing MedMCQA dataset by @monk1337 in https://github.com/huggingface/datasets/pull/4064 Don't do unnecessary list type casting to avoid replacing None values by empty lists by @lhoestq in https://github.com/huggingface/datasets/pull/4282 Enable label alignment for token classification datasets by @lewtun in https://github.com/huggingface/datasets/pull/4277 Fix missing lz4 dependency for tests by @albertvillanova in https://github.com/huggingface/datasets/pull/4295 Altered faiss installation comment by @vishalsrao in https://github.com/huggingface/datasets/pull/4220 Fix CLI run_beam save_infos by @albertvillanova in https://github.com/huggingface/datasets/pull/4294 Add missing faiss import to fix https://github.com/huggingface/datasets/issues/4287 by @alvarobartt in https://github.com/huggingface/datasets/pull/4288 New Contributors @shanyas10 made their first contribution in https://github.com/huggingface/datasets/pull/4240 @apsdehal made their first contribution in https://github.com/huggingface/datasets/pull/4178 @wschella made their first contribution in https://github.com/huggingface/datasets/pull/4239 @TristanThrush made their first contribution in https://github.com/huggingface/datasets/pull/4258 @yanaiela made their first contribution in https://github.com/huggingface/datasets/pull/4153 @mo6zes made their first contribution in https://github.com/huggingface/datasets/pull/4262 @nrajani made their first contribution in https://github.com/huggingface/datasets/pull/4244 @sanchit-gandhi made their first contribution in https://github.com/huggingface/datasets/pull/4266 @cakiki made their first contribution in https://github.com/huggingface/datasets/pull/4207 @monk1337 made their first contribution in https://github.com/huggingface/datasets/pull/4064 @alvarobartt made their first contribution in https://github.com/huggingface/datasets/pull/4288 Full Changelog: https://github.com/huggingface/datasets/compare/2.1.0...2.2.