GitTables 1.7M

Summary GitTables (https://gittables.github.io) is a corpus of currently 1.7M relational tables extracted from CSV files on GitHub, we aim to grow this to at least 10M tables. Each file in this corpus represents a table with the original content (e.g. values and header) as extracted from the corresponding CSV file. Table columns were also annotated with >2K semantic types from Schema.org and DBpedia (provided separately). These column annotations consist of, for example, semantic types, hierarchical relations to other types, and descriptions. We believe GitTables can facilitate many use-cases, among which: Data integration, search and validation. Data visualization and analysis recommendation. Schema analysis and completion for e.g. database or knowledge base design. If you have questions, the paper, documentation, and contact details are provided on the website: https://gittables.github.io. Dataset contents The data is provided in subsets of tables stored in parquet files, each subset corresponds to a term that was used to query GitHub with. The column annotations and other metadata (e.g. URL) are attached to the metadata of the parquet file. In summary, this dataset can be characterized as follows: Statistic Value # tables 1.7M average # columns 25 average # rows 209 # annotated tables (at least 1 column annotation) 1.0M (DBpedia), 1.5M (Schema.org) # unique semantic types 1218 (DBpedia), 924 (Schema.org) Future releases Future releases will include the following: Licensed tables along with their licenses Improved curation (e.g. better parsing, removal of social media content, anonymization) Raw CSV files Responsible use This dataset corresponds to the 1.7M tables used for the analysis in the GitTables paper: https://arxiv.org/abs/2106.07258. Version 0.0.4 of GitTables (the current) contains tables extracted from CSV files from public GitHub repositories. Some tables might therefore not be associated with a permissive license. While the new (fully licensed) set is constructed, GitHub's API can be used to retrieve the license associated with each table based on the URL in the metadata. Please be aware that tables might still exhibit sensitive, harmful or otherwise undesired data. The next release will be curated to mitigate this. If any issue with the content is observed please report this to us using the contact form on https://gittables.github.io so that we can mitigate the issue