This repository includes information about Mega-COV, the dataset first introduced in our recent paper (Mega-COV: A Billion-Scale Dataset of 100+ Languages For COVID-19), currently published on ArXiv (to appear in EACL 2021). We have also published a Medium post about this work. Mega-COV is a billion-scale dataset from Twitter for studying COVID-19. The dataset is diverse (covers 234 countries), longitudinal (goes as back as 2007), multilingual (comes in 100+ languages), and has a significant number of location-tagged tweets (~32M tweets). We release tweet IDs from the dataset, hoping it will be useful for studying various phenomena related to the ongoing pandemic and accelerating viable solutions to associated problems.
We note that the data here is our second version and so it is a more recent version of the initial reelease we had when we first published the manuscript on ArXiv. We will soon add an updated version of the paper, reflecting the current data release.
(a) Left: Cities. Each dot is a city. Contiguous cities of the same color belong to the same country. (b) Right: Point co-ordinates. Each dot is a point co-ordinate (longitude and Latitude) from which at least one tweet was posted.
Word clouds for hashtags in tweets from the top 10 languages in the data. We note that tweets in non-English can still carry English hashtags or employ Latin script.
-
Mega-COV is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License (CC BY-NC-SA 4.0). By using the dataset, you agree to abide by the stipulations in the license, remain in compliance with Twitter’s Terms of Service, and cite the related paper (Mega-COV: A Billion-Scale Dataset of 100+ Languages For COVID-19). (Please see citation section in the end of this README).
-
Data orgnization and download fileslink
We collect Mega-COV from the public domain (Twitter). In compliance with Twitter policy, we do not publish hydrated tweet content. Rather, we only publish publicly available tweet IDs. All Twitter policies, including respect and protection of user privacy, apply. We encourage all researchers who decide to use Mega-COV to review Twitter policy at Twitter policy before they start working with the data. For example, Twitter provides the following policy around use of sensitive information:
You should be careful about using Twitter data to derive or infer potentially sensitive characteristics about Twitter users. Never derive or infer, or store derived or inferred, information about a Twitter user’s:
- Health (including pregnancy)
- Negative financial status or condition
- Political affiliation or beliefs
- Racial or ethnic origin
- Religious or philosophical affiliation or beliefs
- Sex life or sexual orientation
- Trade union membership
- Alleged or actual commission of a crime
- Aggregate analysis of Twitter content that does not store any personal data (for example, user IDs, usernames, and other identifiers) is permitted, provided that the analysis also complies with applicable laws and all parts of the Developer Agreement and Policy.
If you have any questions about this dataset please contact us @ *muhammad.mageed[at]ubc[dot]ca*.
@inproceedings{mageed2020MegaCOV,
title={Mega-COV: A Billion-Scale Dataset of 100+ Languages For COVID-19},
author={Muhammad Abdul-Mageed and AbdelRahim Elmadany and El Moatez Billah Nagoudi and Dinesh Pabbi and Kunal Verma and Rannie Lin},
journal={EACL},
year={2021}
}