TwitStream

Using twitter API to filter and parse through tweets, gather their location data and present said data in a digestible format.

Dependencies

1. Tweepy

2. Geopy

3. matplotlib

4. numpy

5. Basemap

You can install 1 through 4 with

pip install -r requirements.txt

but Basemap is too big to fit in the python package index currently and requires manual installation.

You shouldn't have too much trouble finding it in any package manager.

Instructions

1. Clone this repository wherever required.

2. Add ckey, csecret, akey, asecret in config.py. (If you don't have them yet then register your app in the twitter developer console to receive them. You will need a twitter account with a valid mobile attached to it to use the keys.)

3. In twitstream.py you'll find a filter call with a track list in it. Change it to look for tweets with the specific filters you want.

4. Generate the locations in locations.txt by running twitstream.py.

5. These locations are used in location.py to generate your latitude and longitude in location.py.

6. Map these locations in Basemap by running visualize.py.

Warning

Twitter allows for custom location setting which means a lot of the tweets you parse might not have a location at all or have an invalid location, these are filtered out when generating the coordinates to map, which means a lot of the tweets you parse are of no use.

I've gotten an efficiency of about ~60% (Minor edit: Over a sample of 10k tweets its close to 56%), so if you have a better way of filtering out these tweets please raise an issue to let me know. Any help to better the project would be appreciated.

Examples

Here's a plot of news about rain being tweeted out on 27/9/17. (Sample size of about 500 tweets)

Here's the part where most of the tweets were concentrated. In the southern England.

And here's the weather report from the same day for the same time, courtesy of bing.

As you can see the plot of rain in the united kingdom coincides for the most part with the cloud gathering shown in the weather report. There are a few anomalies which is natural given my twitter filter, but the similarity still holds.

So what's the point of all this again?

I think social media allows for a great deal of information gathering which could prove very beneficial. For example you could track people tweeting about fever/flu like symptoms to track the growth of a disease through a population. This is a pretty small project right now because I'm playing around with the idea on a whim, but the idea itself has a lot of potential.

Feel free to submit issues or PR's if you'd like to help out.