Skip to content

Latest commit

 

History

History
81 lines (62 loc) · 2.75 KB

README.md

File metadata and controls

81 lines (62 loc) · 2.75 KB
Raw

GithubSnapshot

Snapshots multiple organizations GitHub repositories, including wikis, and syncs them to Amazon's S3. The gem also prunes old backups, keeping only the N most recent snapshots for each repository.

Please do keep in mind that we are not setting up an additional remote that is kept in sync. We do a git clone --mirror on each repository and then gzip it with a timestamp. Here is a typical (and reduced for clarity) tree structure of the backup folder/S3 bucket:

github-backups/
├── mozilla
│   ├── brick-201308260300.git.tar.gz
│   ├── brick-201308260300.wiki.git.tar.gz
│   ├── brick-201308270300.git.tar.gz
│   ├── brick-201308270300.wiki.git.tar.gz
│   ├── shumway-201308260300.git.tar.gz
│   ├── shumway-201308260300.wiki.git.tar.gz
│   ├── shumway-201308270300.git.tar.gz
│   ├── shumway-201308270300.wiki.git.tar.gz
└── innvent
    ├── github_snapshot-201308260300.git.tar.gz
    ├── github_snapshot-201308270300.git.tar.gz
    ├── matross-201308260300.git.tar.gz
    ├── matross-201308270300.git.tar.gz
    └── jobs-201308260300.git.tar.gz
    └── jobs-201308270300.git.tar.gz

Installation

github_snapshot should be run as a command line tool, so it makes sense installing it globally

$ gem install github_snapshot

s3cmd should also be installed on the system and properly configured.

Usage

github_snapshot expects a config.yml file on the folder it is run, here is a sample:

username: <github user with read access to the repositories>
password: <github password>
organizations:
  - organization1
  - organization2
s3bucket: <s3 bucket to store the backups>
backup_folder: <backup folder were the repos will be cloned to>
releases_to_keep: <how many releases to keep>
git_clone_timeout: <timeout time for git clone>

Then, simply run the gem's binary:

$ github_snapshot

You could be prompted to check the authenticity for github.com host. You can either say yes during the first script execution or disable strict host key checking for it:

$ echo -e "Host github.com\n\tStrictHostKeyChecking no\n" >> ~/.ssh/config

To-Do

Although this gem was created to serve a very specific purpose at Innvent, here is the roadmap we have in mind:

  • Tests;
  • Backup GitHub issues;
  • Flag based configuration, in addition to the YAML config file;
  • Turn off S3 synchronization;
  • Restore backups to GitHub;

Contributing

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request