Skip to content

Latest commit

 

History

History
32 lines (22 loc) · 3.08 KB

Useful-Tools.md

File metadata and controls

32 lines (22 loc) · 3.08 KB
Raw

Archive Scripts

  • BASC Archiver - Tool that downloads an entire 4chan thread, it's JSON, and all images.
  • Wget - The internet archivist's swiss army knife. Grab any website, and all public data on it for offline viewing (or into WARC format).
  • youtube-dl - Download videos off of YouTube and other streaming sites. [[Here are some recommended commands.|Youtube-dl]]
  • Imgur Album Downloader - To save choice albums for later viewing.
  • Wikiteam - Mediawiki Dump Generator - Archive an entire backup of a MediaWiki-based wiki, all pages, all history, and all images. You can use Special:Import to restore this XML backup.
  • ~~Plowshare - Oh no! Was OP a faggot and gave you 10 split Mediafire links? Not to worry, you can use this automated download script to scrape from any download site in an automated manner. Though sometimes, you'll still have to solve captchas.~~404'ed
  • Bdom's Tumblr Backup - This python script backs up an entire Tumblr blog with all posts and images. Just in case the author decides to delete their blog...
    • Unfortunately, it doesn't grab a list of reblogs/comments or the Tumblr theme. Might be a good project to work on.

Quick and dirty archive snapshot

Just click a bookmarklet to archive a page for later! Great for source links and Google Cache, because you can bet that they will be gone within 5 years.

  • Archive.today - An improved version of WebCite, optimized for pesky "Web 2.0" sites that dump a lot of extra CSS and other content that is tricky to archive.
  • Webcite - The original tried-and-true website snapshot archiver, for research paper citations.

Markdown Conversion

  • Pandoc - The Swiss Army knife of markup language converters. Convert anything to Markdown, and convert markdown to anything. Great for turning HTML into a readable format.
  • gdoc2md - A quick and dirty script used to convert a Google Doc into Markdown. Great for importing collaborative projects.

Regular Expressions

  • Regexr - Create and share Regular Expressions for everyone to use. Great for learning how to make them.

Programming Guidelines

  • Use the GPLv3 for programs/scripts. - This ensures that the source code of our programs stay open-source, and any derivatives are open source.
  • Use the LGPLv3 for libraries. - This ensures that the source code of our libraries stay open source. Unlike the GPL, however, the LGPL allows statically linked (import library) programs to use any license they want.
    • The only exception is the BASC-py4chan wrapper, which uses the WTFPL. This is a humorous tradition inherited in honor of Edgeworth E. Euler.