A project to look at public comments to the FCC on net neutrality.
The FCC takes public comment through its Electronic Comment Filing Systems and also makes available a nice API for the ECFS, at least for now. To use it, you'll need to sign up for an API key.
You know you want them. Here's how to get them as a series of .json files with 1000 filings each.
python3 python/get_filings.py --limit 1000 --max 10000 --start 500000 -o my_directoryOK, now fire up ipython and read them and concatenate them into an array.
import glob
import json
import os
def read_filings(output_dir):
filings = []
path = os.path.join(output_dir, '*.json')
for filename in glob.iglob(path):
print(filename)
with open(filename, encoding='utf-8') as f:
filings += json.load(f)['filings']
return filings
filings = read_filings('data/may_15')