Home
Some lists to seed your server:
- Alexa500 (composed on 2014-07-28) - 395 sites, useful for starting a feasible server with very popular sites
- HTTPSEverywhere Atlas (composed on 2014-07-28) 10835 sites, useful for performance testing only as most sites are probably not very popular
(These lists were composed using Alexa topsites and HTTPSEverywhere Atlas testing for example.com:443,2 and www.example.com:443,2 schemes and have all sites removed which didn't support HTTPS.)
Import the list with:
python notary_util/file2db.py -s THE_LIST.txt
(the -s parameter indicates that sites are imported without keys and timestamps)
Note that file2db.py prints a huge MySQL command and minimal systems may run out of memory (Raspberry Pi Model B with 512MB RAM can only parse 500-900 lines per import). Thus check if the import was successful by comparing the number of imported sites with the number of sites in your list:
python notary_util/list_services.py list_services.txt && wc -l list_services.txt
If the number doesn't match then something went wrong and you probably have to split the list into multiple files:
split HTTPSEverywhere_20140728.txt -l 500 -d HTTPSEverywhere_part
for i in HTTPSEverywhere_part*; do python notary_util/file2db.py -s $i; done
Scanning the HTTPSEverywhere_20140728 list (~10k sites) with 20 sites per 10 seconds took:
- 150kbit/s average bandwidth (up/down)
- for about 1 hour
- producing 60MB of traffic
- and eventually taking up 4MB of disk space