Support asymmetric backups

[fd0]

This issue should collect use cases for asymmetric backups. In this situation, restic is able to efficiently create new backups, but unable to decrypt/restore and/or modify old backups. Please add your use case to this issue if you have one. I think we have enough use cases, thanks!

Summary (2018-04-28)

At the moment, restic (mostly) interacts with "dumb" storage (local, s3, b2, gcs, azure, everything except for the REST server with --append-only). restic can save, list, get and delete data in a backend. This is required for a backup, so the needed credentials to access the backend need to be present. On the upside, restic can use almost any storage, there are very few restrictions. On the downside, once attackers gain access to a server, they can easily extract the credentials for accessing the backend and the restic password, giving them all possibilities: Decrypt historical data from the repository, modify data, delete the whole repository.

When we add asymmetric cryptography, the only difference for attackers in such a situation is that they cannot decrypt historical data from the repository. Everything else, especially deleting all the data, is still possible. So "just add asymmetric crypto" is not the whole story.

The other idea is to not access the "dumb" storage directly, but indirectly via a custom server implementation. We've played around with this idea and added the --append-only option for the REST server, which can be seen as an "adapter" to access the "dumb" storage at the local harddisk.

The only exception from the first paragraph of this summary, and an implementation of the "adapter", is the rclone backend: It can be accessed e.g. via SSH (restic -o rclone.program='ssh user@server', with a hard-coded call to rclone via ForceCommand in .ssh/authorized_keys for the SSH key the user logs in). The cloud access credentials reside on the server, the user and machine running restic won't have access to those. If --append-only is specified in the call to rclone, data can only be added.

Having "non-dumb" storage alone won't help against attackers reading data from the repository (at least not without changing the repo format), but will prevent deleting all data in the repo.

So, in conclusion, to defend best against attackers taking over a server that uses restic for backups, I think we would need to implement both (non-dumb storage and asymmetric crypto). That's a long-term goal :)

[cfcs]

A brief recap from the previous discussion:

• Useful for backups of email servers or remote backups of log data in the event that such a server is compromised. Sensitive data that has been shredded from the server may be present in the backup, and it may be useful to be able to deny an attacker access to such historical data. It was simultaneously discussed that it could be useful to have a "blob network server" with a capability system that allows you to configure append-only / read-only / read-write access as appropriate. Having asymmetric cryptography in place would remove the need for having yet another set of symmetric keys for issuing capabilities, and would likely simplify the implementation of a capability system as well.
• In a setting with several machines, one backup key can be used to access several backup datasets without having to manage a multitude of symmetric keys. If a cryptosystem such as Daniel Bernstein's Curve25519 is used, such a key may be derived from a passphrase, meaning you can use one master passphrase to manage backups created by a (potentially large) number of different security domains. Although something similar can of course be achieved with n symmetric keys and an encrypted key storage.

[fw42]

@heipei FYI you might want to subscribe to this issue

[ThomasWaldmann]

well, the most obvious and pressing use case is not having the backups erased by an attacker who has broken into your production system (backup client) and knows how to use restic from there.

[fd0]

That's hardly possible just by using asymmetric crypto. The point here is that old backup contents (not necessarily metadata) should not be available to an attacker who has broken into a server.

[cfcs]

Data confidentiality (write-only) would be achievable for these scenarios with the implementation of asymmetric crypto.

Data integrity is a slightly different beast:
While you can use asymmetric cryptography (and a one-time signature scheme) to prove that a given dataset has not been altered, you cannot prevent its complete removal or replacement. That is a problem that requires a storage system that is append-only (and read-only, for some parts).
Thanks to discretionary access control that is relatively straightforward to implement on *nix with a backend such as ssh (using commands like chmod, chown, chattr +i, chattr +a to configure the access policies). append-only backups don't even need to be encrypted to prevent attackers from reading them (that's part of what rsyslog does, for example), but that suddenly makes the backup server an interesting target, and cloning sensitive data to more devices doesn't exactly improve the odds of maintaining data confidentiality.

Having asymmetric crypto implemented would enable people to do this kind of backups on dumb, untrusted systems, one of the things restic is all about (apart from the improved deduplication and all the other nice features). I hope this elaboration clarifies my point a bit.

[JensRantil]

This is of interest to us for the two cases described in #187 (comment). I'm subscribing to this thread to keep an eye on this.

[lorenzb]

Regarding data integrity: AFAIK, you can achieve append-only behaviour on Amazon S3 by only granting your IAM credentials the PutObject and GetObject permissions, but withholding the DeleteObject permission.

Regarding data confidentiality, I wanted to mention an attack scenario that is enabled by the use of symmetric crypto:
If an attacker has control over a victim's user account at a point in time where the victim uses restic to perform a backup, he can:

• steal the victim's restic key
• steal the victim's IAM credentials/sftp login/...

The attacker can then immediately wipe any trace of his attack from the victim's system making detection less likely. Since he has stolen the restic key and the credentials for the remote storage system, the victim will conveniently deliver any (backed-up) future data to him: Our attacker only needs to download and decrypt new backups from the remote storage system from time to time.

Asymmetric crypto could help prevent this by allowing the victim to store the key for restoring backups offline.

[JensRantil]

Regarding data integrity: AFAIK, you can achieve append-only behaviour on Amazon S3 by only granting your IAM credentials the PutObject and GetObject permissions, but withholding the DeleteObject permission.

Unfortunately PutObject also gives you privileges to overwrite a previously uploaded file. So, maybe it's not full data integrity?

[viric]

Asymmetric crypto would allow also the usage of an OpenPGP key like yubikey or so.

[viric]

Today I realized that I don't really want support for asymmetric encryption in restic, but support to NOT store the key in the repository. I understand that using asymmetric encryption efficiently would mean that restic can upload new backups without the ability to read the repository, which is quite tricky.

So, for me, it would be fine if I could handle the symmetric key without restic, and it were not uploaded in any way to the repository, no matter what complex KDF is used.

[egalanos]

My use case is as a system administrator for a number of servers.

Asymmetric backups are the only way to protect the backups in the scenario of a server compromise where an attacker maliciously destroys (or ransomware encrypts) all the data (including the backups).

This needs server side support through either an append only storage layer or a server daemon similar to the way rdiff-backup has the --restrict-update-only option. Currently I work around this by using read-only snapshots of the backup repository on the backup server (accessed via sftp).

[cfcs]

(Perhaps relevant): Append-only can be achieved in Linux through the the append-only flag on a directory (which disable unlinking) along with the immutable flag on the files. The commands responsible for setting those flags are chattr +a /path/to/directory and chattr +i /path/to/directory/myfile01, respectively.

[anarcat]

my use case here is #533 - unattended backups. as stated there, asymmetric crypto is only one of the ways of doing this, but it seems like the first obvious solution to the problem.

[michbsd]

In a scenario where the repo is on a remote server - only local commands on the repo should be able to forget/prune.

The restic backup, should connect with a unique key for that system with restore / backup privileges only.

[bherila]

In a scenario where the repo is on a remote server - only local commands on the repo should be able to forget/prune.

This should be accomplished by restricting access to delete/modify files on the repo level. I think it's out of scope (and not even secure) for restic to manage these permissions. After all, someone could delete the repo or even the keys, rendering the whole repo useless, irrespective of whether that action were allowed by the restic client.

[wojas]

Regarding preventing backup data from being destroyed by someone that hacks the server: rest-server has recently gained an "append only mode" with PR restic/rest-server#28 that prevents exactly this.

[linsomniac]

My use case is having many systems back up to the same repository, getting the advantage of deduplication across all those machines, but a compromise of one system (and it's backup script) not allowing the attacker to read the backups of the other systems.

[alexxroche]

The feature I'm looking for is to have "backup" keys that will enable a system to write(backup) and read(restore) but can not do any admin, (such as prune, add keys or even see the existence of other keys, (users) or snapshots not associated with $backup_key ). (Though a side-channel attack might be possible by comparing backup times, it does not matter to me if they can determine the existence of data, only that they can not ransomware my data and can't view other users.) I would expect the holder of a backup-(only)-key to be able to roll their own passphrase(s) forward. So unlike michbsd's request I would be able to admin from a non-local machine with a privileged key. (Having used SELinux for years I'm now a fan of the granularity of M.A.C.) Thanks for reading. (Sorry if this should have its own issue.) With this feature #ResticKillsRansomware

[alphapapa]

With this feature #ResticKillsRansomware

In general, pull-oriented backups (as opposed to push-oriented ones) solve ransomware, right? :)

[alexxroche]

Possibly but that then gives remote access and adds another attack vector. How I've designed it my backup server is for preserving data and should never have access to my production environment. Clear demarcation of functional domains.

[fd0]

@andrewchambers thanks for writing it down, I'm very interested how the actual implementation looks like. The blog post left some interesting bits open :)

I like that there will be another contender in the space of free softwar backup programs, giving users more options is always great!

[anarcat]

So an interesting parallel can be made with the two git repository encryption mechanisms.

On the one side you have git-crypt: this uses git smudge/clean filters to (respectively) encrypt/decrypt files between the blob storage and the checked out copy. This works well and is fairly optimal, but it has one glaring hole: the git commits themselves are not encrypted, only the contents of the blobs, which means that filenames, commitlogs, authors, dates and other metadata is all store in the clear. That's a no-go for many use cases and is effective only when you have a public repository where (say) you want to encrypt some bits (but not all).

On the other side you have git-remote-gcrypt: that uses the git remote helpers protocol to encrypts everything that is sent on the server. But that is very inefficient, because the entire repository is re-encrypted on each run, due to the way special remotes work.

Now, those are git-specific implementation challenges, but I think they map well into the problems you might get here. Maybe i'm totally out of my depth here and this parallel is irrelevant, but I figured it might be of interest here...

[cdhowie]

As an aside, there is a middle ground that could currently be implemented (probably) rather easily: allow keys to be stored outside of the repository.

One of the attack vectors being addressed is that the attacker gets his hands on a key password, and then (since the keys are stored with the repository) he can decrypt a key with ease.

What if we allow specification of a separate key directory, where the key files are stored? This directory could be stored locally on each machine that needs to do backups, and it could itself be backed up to a different cloud provider, or even a QR code (~500 bytes is plenty small to be QR-encoded) for cold offline storage in a safety deposit box, for example.

If the encrypted keys never touch a cloud provider, the attack vector goes away completely. The keys would have to be compromised from the physical premises or exfiltrated with malware, for example.

This can be done on Restic already if a local copy of the repository is kept -- just exclude the keys directory from being synced to the untrusted remote when running rclone. This cannot be done if there is no local copy and restic directly interacts with the untrusted remote.

[yookoala]

I think we should apply the single responsibility principle and break things down into 2 tasks:

1. Keep data safe from decryption.
2. Keep data safe from unauthorized delete action.

They are 2 different aspects of data safety. Technically they don't have to rely on each other.

For (1), obviously we can "simply add asymmetric encryption support". For (2), I believe there are many possible solutions (for example, as mentioned above, append-only S3 setup).

[dp-github]

Necro on a very old issue, but I'd like to suggest a use case for the opposite type of asymmetric backup: Restic can read old snapshots but cannot read recent ones or write to any of them.

The use case here is very simple: We want to revoke a user's access to the repository. (And we want to do it in a way that is consistent with the restic threat model: continuing to "store backups in a location that is not completely trusted, e.g. a shared system where others can potentially access the files")

Currently, there is no way to do this, short of replacing the entire repository. restic key remove is ineffective, because presumably that user already retrieved the master key to the repository and therefore has access to all its data.

But, under an asymmetric approach, users with valid password could begin writing new data into the repository using a new key. (Note that I am not suggesting all data must be rewritten using the new key - multiple master keys could be used in the same repo).

A user with a revoked password has the master key required to access old data, but this is no great harm since he previously had access to that data and presumably made a copy of it. But this user cannot retrieve the newer master key required to read any new data added to the repository after his password was revoked.

Note that this approach could be enhanced with a restic key migrate function to allow packs and blobs to be migrated from an old masterkey to a newer masterkey.

[mathstuf]

Keep data safe from unauthorized delete action.

Note that anyone with access to the restic repo probably also has permissions to rm files willy-nilly. Not much restic can do about that though (chattr +i needs elevated permissions). I don't know how useful this is other than denying prune availability to unauthorized keys (because they can't know if a blob is needed by snapshots they can't read).

Restic can read old snapshots but cannot read recent ones or write to any of them.

This sounds like a separate issue (though one that intrigues me). Out of curiosity, is it something that can be added to any asymmetric support later or does it need to be considered from the beginning (at the restic code level; needing to redo a repository to add this feature to it makes sense to me)?

[andrewchambers]

fwiw I have released my backup tool I mentioned earlier which addresses these issues: https://github.com/andrewchambers/bupstash

[gsauthof]

Keep data safe from unauthorized delete action.

Note that anyone with access to the restic repo probably also has permissions to rm files willy-nilly. Not much restic can do about that though (chattr +i needs elevated permissions). I don't know how useful this is other than denying prune availability to unauthorized keys (because they can't know if a blob is needed by snapshots they can't read).

No, this isn't true in general. It really depends on where the repository is located. For example, for S3 or other object storage buckets (which restic supports directly and/or via rclone) it's trivial to give a backup user only write/create permissions but not remove/purge permissions. That means that auch a user is able to create new objects but can't delete existing ones. With e.g. Backblaze b2 object storage it's also possible to write new versions of an existing object but older versions are kept around (until the object is explicitly deleted, given the user has the appropriate permission), by default (the bucket administrator can configure the retention policy).

[mathstuf]

No, this isn't true in general. It really depends on where the repository is located.

And as such, is a guarantee of the backend in use and not restic itself, no? Restic can certainly provide documentation on how to set up repositories to be secure against deletion (such as B2's bucket creation options), but I would expect that that is the extent of it. Of course, restic might have some control with certain backends and appropriate controls can be added there. But there's nothing restic can do to provide it in the generic backup case AFAIK.

[klausenbusk]

For example, for S3 or other object storage buckets (which restic supports directly and/or via rclone) it's trivial to give a backup user only write/create permissions but not remove/purge permissions.

Please be aware: At least for S3 writing or overwriting a file is the same, so a attacker could just overwrite all the files with zero byte files, which mean you need to use versioning or object lock to be safe.

[gsauthof]

@mathstuf wrote:

And as such, is a guarantee of the backend in use and not restic itself, no? [..] how to set up repositories to be secure against deletion (such as B2's bucket creation options), but I would expect that that is the extent of it. [..] But there's nothing restic can do to provide it in the generic backup case AFAIK.

No and no. Your expectation is wrong.

If you read the summary in this issue from restic's author, you see that restic currently requires delete privileges for normal backup operation (with allmost all storage backends). This is also the reason why this issue was entered and why it is still open.

There are related issues that state in detail how restic fails with e.g. B2 if a target bucket was created without that privilege.

In the meantime the documentation was likely updated to note this point, as well.

See also, for example:

• b2: Add optional secondary key for locks #2887
• Use b2_hide_file instead of b2_delete_file_version #2134
• B2 write-only application keys do not work? #2293 (comment)

[lgommans]

@andrewchambers Does your overwrite/delete prevention in bupstash work the same way as is implemented by restic/rest-server? Or are there docs on how this is implemented? I was browsing the docs, particularly the access controls page, but didn't spot how it works technically. Given that borg claims the same but its effect is quite limited, I'm trying to check what other backup software there is out there. Also still trying to find details on rest-server for restic but it looks I might have to go and read the source code.

[andrewchambers]

@lgommans it blocks any operations that would cause deletion.

[d4nyl0]

It should be enough for restic to accept an external command to encrypt data. In this way anyone could use GPG to encrypt with their own public key.
Obviously during a restore, restic should be able to use an external command to decrypt data, in this case GPG with private key.

[yan12125]

(From #187 (comment))

which mean you need to use versioning or object lock to be safe.

Restic >= 0.13 works with S3 object locks (#2202). Maybe data integrity can be guaranteed for AWS S3 now?