Merge pull request #2 from ZAdamMac/dev

Merging Dev to Master for the 1.0.0 release!

CODE_OF_CONDUCT.md

@@ -0,0 +1,46 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at Tap_Dev@psavlabs.com. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

CONTRIBUTING.md

@@ -0,0 +1,36 @@
+# Welcome to the Tarnished Tale Contributor Guidelines!
+Hey, thanks for checking us out! Tapestry's renaissance phase of development is ambitious at best, which means we're always looking for contributors to come in and help us make a best effort at producing something truly useful - a secure, trust-minimized backup tool made with the modern internet and the user in mind.
+
+First off, let me just say that any contribution matters - from pull-requesting in new functions, to requesting new features, or even just reporting your crashes and bugs. Right now a lot of development is underway. A lot of our framework is super minimal. Tapestry doesn't have automated crash and bug reporting, and its unlikely it ever will. I'm just one guy with a few spare hours in the evening trying to put this thing together. Everything you contribute helps.
+
+Secondly, we do have a few resources for the eager:
+- [**Discord Server:**](https://discord.gg/56msGFT) easily the fastest way to join in the conversation or ask a question;
+- [Email Development Team](mailto:tapdev@psavlabs.com);
+- and, of course, this guide!
+
+# Development Team
+While everyone who contributes to the project is part of our development team, the capital-letters Development Team currently consists of github user ZAdamMac.
+
+# Feature Requests
+We're always happy to have a look at requests for new features. They further our quest for maximum utility and usability.
+
+## Filing a Feature Request
+If you have a feature request for Tapestry, go ahead and open a new issue at this repo. Be sure to head the title of the request with `[RFC]`. The development team will cross-post your request in announcements to the discord and any other contact trains we happen to be using at the time, opening the request up for commentary by other members. **A minimum of 30 days later**, the same group will decide whether or not to include the feature, or some version of it, into the development process. If the feature is complicated or discussion over it is lively, this decision process may be extended as necessary to allow the matter to resolve.
+
+# Bug Reports
+We kind of can't overstate the importance of providing your bug reports to us. It's so important we've made a template for them. When Tapestry crashes on your system, we have no idea what happened or why. Please be encouraged to file an issue and fill out the bugs form. Super important stuff.
+
+# Contributing with Code: The Pull Request!
+Here at Tapestry we use Fork and Request. If you want to contribute to the development process with your code, the preferred method is to first fork the repo, make your changes there, and then submit a pull request. For obvious reasons all submissions are subject to review. To make this whole process easier, be sure to use nice, clear, descriptive code. Provide documentation or comments as needed.
+
+In general though, we're super happy to have any help that we can. People who contribute with code are being kept track of and will be included in the credits.
+
+## Testing
+Tapestry is tested through the use of functional tests - these tests can be found in the testing folder of the main repo, under `functional-tests.py` and the corresponding documentation. Passing the functional tests merely shows your changes aren't breaking to the core behaviour of Tapestry - it will not test the effectiveness of your new features.
+
+If your pull request adds features, be sure to add the corresponding tests to functional-tests.py and submit the new copy with your PR. Also, your PR should include the output of at least one successful test log.
+
+# Contributing with Coins
+Lastely, as a hobby project, we're obviously working with the bare minimum funding possible - luckily, this sort of thing also doesn't take much. If you'd like to donate to keep Tapestry's development alive, the current money repo is [ko-fi.com](https://ko-fi.com/PSavLabs). Obviously, if a permanent developer team is adopted, this model will have to change.
+
+At present, we simply have no way for you to contribute with cryptocurrencies. 

DOCUMENTATION.md

@@ -0,0 +1,111 @@
+# Tapestry Specialist Backup Tool
+*User Documentation, prepared for Version 1.0*
+
+## Full System Requirements
+Tapestry is a reasonably lightweight and flexible script in its essence, but it does involve some basic requirements.
+
+**Suggested Minimum Hardware Requirements**
+- 6GB RAM (or 1.5*Blocksize, if changing blocksize)
+- 3.0 GHz, 64-Bit Dual-Core Processor (or equivalent)
+- 10 GB or more unusued Hard Drive Space
+
+**Software Requirements**
+- 64-bit Linux/Unix-Based OS (Recommended)
+- Python v3.7
+- Python-GnuPG, v.0.4.2 or later
+- GnuPG 2.1.11
+
+### Other Considerations
+Tapestry runs are fairly long - on the order of twenty minutes per default-sized block, depending on your system resources and the amount of other processes running concurrently. Accordingly it's considered helpful to use cron jobs or other automation in order to run the backup overnight or during other periods of low-activity uptime.
+
+It is currently required due to software limitations that the recent version of GnuPG is installed as the primary instance. That is to say, a call to `gpg` should instantiate the latest version of it installed.
+
+## Complete Explanation of Configuration
+Tapestry stores its user-adjustable configuration files in `tapestry.cfg` which it expects to find in the same directory. If you are testing against a development version of tapestry, it will instead look for `tapestry-test.cfg` in the same area.
+
+*As of version 1.0 the use of the --init and --setup functions to modify or generate the configuration files was considered deprecated and future versions will not include like function. When modifying configuration the current best practice is to use the text editor of choice.*
+
+### Environment Variables
+
+|Option|Default|Use|
+|---|---|---|
+|**UID**|Set by user during setup()| the expected user identifier, used to autogenerate paths during setup. As setup is soon to be deprecated, the UID tag likely will be as well.
+|**compID**|Set by user during setup()| The "label" to assign to backups generated using this particular tapestry instance. Suggested use is either your organization's workstation identifier or the system hostname.
+|**blocksize**|4096| The size of the expected output files, before compression, in MB. Suggested value depends on intended storage medium. Since files greater than this size value will not be backed up, setting this value is a matter of personal requirement.|
+|**expected fp**|set by --genkey|The fingerprint of the **disaster recovery key** incoded as a string. This is to be clarified in a future refactor|
+|**sign by default**|true|Boolean value as to whether or not the system should use signing. Set to true by default. Highly recommended not to disable it for any reason.|
+|**signing fp**|None|Set by the user, this is the hex string Fingerprint of the intended signing key. Should be different than the disaster recovery key, preferably specific to the user.|
+|**recovery path**|`/media/`|The directory used to determine the mount point or other location of the .tap files expected by the recovery mode./
+|**output path**|No Default|The directory to which tapestry is to deliver the final packaged .tap files, and other outputs like the skipped file log or keys exported during --genKey|
+|**keysize**|2048|The size of key to generate during --genKey and as part of first time setup. 2048 is the minimum viable, and therefore sane, default.
+|**use compression**|True|Toggles the use of Tapestry's built-in bz2 compression handler. If set to true, blocks are compressed before encrypting to keep them under the blocksize.|
+|**compression level**|2|A value from 1-9 indicating the number of bz2 compression passes to be used. Experimentation is required for different blocksizes to determine the minimum viable value. 9 passes is maximally efficient, but also takes considerable time, especially on larger blocksizes.|
+
+### Additional Categories
+Additionally, the user will find categories for windows and linux options, indicating they are either "default" or "additional" locations for backup. Any number of these definitions can be included at the user's discretion, so long as each option label is unique. When doing this it is desirable to set equivalent paths for both OS varieties, but as Windows support was broken in 0.3.0, the windows categories are not formally significant.
+
+## Runtime Arguments
+Tapestry supports the following arguments at runtime:
+
+|argument|function|
+|---|---|
+|--setup|Drops to the soon-to-be-deprecated setup menu system. The setup menu is badly out of date and crudely designed. It is better to modify config directly using a text editor.|
+|--genKey|Generate a new RSA public/private keypair designed to be used as the Disaster Recovery Key. In a pinch this could also be used to generate a signing key, but there are better ways to do that.|
+|--inc|Performs an "inclusive run", adding all of the "additional locations" categories to the work list at runtime. Provides non-granular differentation between "quick" and "complete" backups.|
+|--rcv|Places the script in recovery mode, checking its recovery path for .tap files and their associated .sigs and recovering them programatically.
+|--debug|Increases the verbosity of both Tapestry and its gpg callbacks for light debugging purposes|
+
+If no runtime arguments are provided the program assumes you intended to do a "basic build", and runs the backup routine using only the relevant "default locations" list.
+
+## Key Security
+Tapestry relies on a two-asymmetric-key system for its protection, as a mechanism to eliminate the need for trust between the user and their storage solution. Tapestry is currently designed to produce only its own key automatically - for the moment it is taken as read that the user would know how to develop a signing key. Specific instructions for signing key generation can be found in the GnuPG manpage or their online documentation. For the purposes of this section, it will be enough to concern the active and passive key security considerations.
+
+### Disaster Recovery Key versus Signing Key
+Tapestry relies on two different keys, hereafter referred to as the Disaster Recovery Key and the Signing Key. It uses these keys for two separate operations: encryption and decryption of the backup files, and signing/signature verification of the same. Seperate keys are used to allow positive identification of the user or terminal (depending on use case) which generated the backup. Signed backups resist tampering in ways that unsigned backups would not - to forge a tapestry backup would require knowledge of the private side of the signing key. Without the signing key, it would be as simple as knowing the public side of the disaster recovery key.
+
+Why use assymetric cryptography at all, when "keyphrase"-based symmetric cryptography would have sufficed? Distribution. 500 systems could comfortably share one disaster recovery key, with each system holding only the public key in its respective keyring, and a trusted admin or other "super-user" in posession of the private side of the key.
+
+### Key Size and Passphrase
+By default, Tapestry creates a 2048-bit key when prompted to. This is the smallest common-size key we believe to be reasonably secure. If desired, this figure can be increased, though this is not recommended as it would impact both key generation and overall cryptographic operation time. With cryptography being the second-most computationally-expensive part of the system, and 4096-bit keys being excessive, we have settled on 2048 bit.
+
+All keys use should be protected by a strong passphrase of at least 24 characters.
+
+### Key Storage
+Tapestry expects to find the keys it is looking for on the default gnupg keyring, found under `~/.gnupg/`. There is currently no plan in the works to change this.
+
+However, we have some concerns that private keys stored directly in the keyring may not be secure under all circumstances, though we are, of course, professionally paranoid. For the home user, keyring storage is fine. If you are a small business or other organization, it may be preferable to use a smart-card based system. Your existing 2FA solution, such as a Yubikey, may provide this functionality in addition to its normal use. Configuration in this way is beyond the scope of this documentation but highly recommended - the developers have been treating their keys in this manner for some time and find the process highly seamless.
+
+Additionally, it is important to keep a master copy of the disaster recovery key offline and secure at all times as a backup. If you should happen to lose this key, your backups are unrecoverable.
+
+### Passphrase Security
+Tapestry Project endorses long passphrases punctuated randomly and including numerals for key passphrases.
+
+Tapestry itself never handles a single passphrase you provide it, either for recovery decryption or signing of backups, or indeed for key generation. This is the reason for the requirement for a recent version of GnuPG to be installed. Tapestry provides the command for the operation to GPG without a passphrase, prompting newer versions of GPG to respond by invoking the pinentry program they are configured to use.
+
+### Other Key Management Considerations
+Depending on the nature of your organization it may be desireable to have your Disaster Recovery Key expire - of course, as you are in posession of the master key, you can extend these expiries indefinitely. An expired card cannot be used to encrypt new messages, but it can still be used to decrypt messages sent to it.
+
+A similar principal of operation exists for revocation certificates. At present, it is not programatically possible for Tapestry to neatly generate a revocation certificate - you should perform this operation yourself as soon as a new key is generated and again, store it as securely as possible. In the event your private key becomes compromised, you can issue the revocation certificate by pushing it to any and all keyservers you are using, which prevents new backups being created which would be decodable by that key.
+
+While we encourage keeping a copy of the master DR key's private side offsite in case of catastrophe, we do not encourage using online services to do so.
+
+For home users it may be excessive, but as an organization I highly recommend generating keys on an offline machine, perhaps even from a known-clean live boot environment.
+
+## Selecting your "Locations"
+Tapestry treats every location defined in its configuration file as the top of an os.walk() command. This means, in practical terms, that everything in every subdirectory of that location will be backed up. Therefore, it is important to consider if any symbolic links are going to be followed that may end up with unintended consequences.
+
+The specific locations you select are entirely up to you. At time of writing I personally use the documents and photos default folders in my default locations list, with my additional locations list including videos, music, and a subset of the hidden configuration directories.
+
+## Example Runsheet: First-Time Setup
+0. Download the latest version of tapestry from the github repo (at time of writing, 0.3.1), and verify it against its own signature. To do this you will need a copy of [this key](https://pgp.mit.edu/pks/lookup?op=vindex&search=0xF373FF4B43FC742F).
+1. Unpack the verified tar of Tapestry. It should contain Tapestry.py, README.md, DOCUMENTATION.md (versions 1.0 and up), and an example tapestry.cfg file.
+2. Open up tapestry.cfg in your text editor of choice and adjust the configuration as follows:
+ - Refer to the configuration table above for explanations of the environment variables, and;
+ - Define default and additional locations as required by your backup model.
+3. Run tapestry.py using the standard methods for invoking a python script, with the `--genKey` flag. This will be a command similar to:
+```
+python3 tapestry.py --genKey --inc
+```
+4. Follow the onscreen directions to generate your first disaster-recovery key.
+5. (Optional) Take the time now to generate a signing key if you don't already have one, as well as the revocation certificates for both. Be sure to back these up and store them securely.
+6. Store your keyfiles and the generated inclusive backup safely and securely.