add support for append-only option

[bricewge]

I didn't find a way to use the option --append-only from the last rest-server in this plugin.
Having feature parity between server and plugin would be great.

[fd0]

Indeed, I need to find out how to add this to the caddy plugin.

[MatthewVance]

I've been doing some research to see if I could determine how to do this. Looking through @mholt's docs, I think the answer on how to add support for the rest-server's append-only flag may be in https://github.com/mholt/caddy/wiki/Writing-a-Plugin:-Directives. I wish I could be of more help, I'm just not familiar enough with Golang or Caddy plugins at this point to contribute more.

[mholt]

Yes, that's on the right track. There'll probably be some keyword added for the Caddyfile so that restic would be invoked with append-only. :)

[MatthewVance]

Thanks for chiming in. That makes sense. Can you point me to any existing simple plugin examples I could reference to see how it was implemented in other code? I'm a novice programmer so having examples to follow help 😄

[mholt]

Sure, pick any here: https://github.com/mholt/caddy/tree/master/caddyhttp - these are all the standard HTTP plugins.

A pretty simple one is ext: https://github.com/mholt/caddy/blob/master/caddyhttp/extensions/setup.go

Or mime, a little more advanced: https://github.com/mholt/caddy/blob/master/caddyhttp/mime/setup.go

[MatthewVance]

Thanks again. Now to find time to dive in and learn.

[fd0]

FYI, Nick (@ncw) recently added an option to provide a restic REST server implementation in rclone (a cool cloud sync tool which supports many different services) here: rclone/rclone#2116

Among others, he's written a new HTTP handler which serves the restic REST protocol, independent of the one from the rest-server, starting here: https://github.com/ncw/rclone/pull/2116/files#diff-2bd148b7531377d2d2cfe6cd70c0c74fR192

It may be valuable to integrate the code (if the license permits it) into caddy, Nick's implementation is much more configurable and portable as far as I can see.

Bonus point: The append only mode can also be integrated into rclone afterwards ;)

[MatthewVance]

Good to know The code appears to be under MIT so licensing should not be an issue if someone wanted to pursue this. https://github.com/ncw/rclone/blob/master/COPYING

…

On Mar 13, 2018 3:32 AM, "Alexander Neumann" ***@***.***> wrote: FYI, Nick ***@***.*** <https://github.com/ncw>) recently added an option to provide a restic REST server implementation in rclone (a cool cloud sync tool which supports many different services) here: rclone/rclone#2116 <rclone/rclone#2116> Among others, he's written a new HTTP handler which servers the restic REST protocol, independent of the one from the rest-server, starting here: https://github.com/ncw/rclone/pull/2116/files#diff- 2bd148b7531377d2d2cfe6cd70c0c74fR192 It may be valuable to integrate the code (if the license permits it) into caddy, Nick's implementation is much more configurable and portable as far as I can see. Bonus point: The append only mode can also be integrated into rclone afterwards ;) — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACjDujT4_wWAllSG2I-nrUA3EGKO90KWks5td4QwgaJpZM4QKEI4> .

[mholt]

@fd0 That's great!

Do you know if those changes allow us to proxy a backup? i.e. if the backup is happening on an untrusted computer, and we need to keep the cloud storage credentials locally on the proxy server which we trust?

[fd0]

Do you know if those changes allow us to proxy a backup? i.e. if the backup is happening on an untrusted computer, and we need to keep the cloud storage credentials locally on the proxy server which we trust?

Yes, that's one of the use cases. You could, for example, run rclone serve restic on a server, and use the provided REST service with restic over the network (be sure to configure rclone to require authentication). You could also use ssh to start a rclone remotely like this:

$ restic -o rclone.command="ssh user@server rclone serve restic --stdio b2:bucket/path" --repository rclone:foo backup $HOME

The rclone/ssh process is automatically started and stopped by restic, and communication happens over HTTP2 over stdin/stdout, so there's no network socket involved. This will be supported when restic/restic#1657 is merged, but you can check it out already and test it with the master version of rclone.

I'd love to get some feedback btw.

Either way, the rclone process is the only process with access to the backend credentials, so it's exactly what you're asking.

You could even hard-code the rclone call in a force_command for an SSH key that's installed for the user on the server, so the user does not have any control over which remote for rclone is used, and you can also restrict the user to a subdir in e.g. a B2 bucket.

All of that isn't documented yet (but still very exciting!)

[mholt]

@fd0 I finally had a chance to sit down and try this today, and it works so far! Thank you so much to both you and @ncw for making this possible. This is extremely awesome. Looking forward to when both programs have it in their stable release!

So, I dunno, in my mind, since credentials don't have to be given to the machine doing the backup, does that eliminate the need for the original feature request (--append-only in the caddy plugin)? If so, we can close this.

[MatthewVance]

That's great you had a chance to try this and confirm it works. I haven't had a chance to look into adding append only support. Regarding your question, I'm not sure. Here's the description from the current rest server: The --append-only mode allows creation of new backups but prevents deletion and modification of existing backups. This can be useful when backing up systems that have a potential of being hacked. I think the proxy model would help when dealing with untrusted computers. Supporting this flag would give an another way to mitigate the threat for people who prefer not to go through a proxy. People who care about all this may prefer the added benefit the proxy method offers making this point less of a concern. I'm putting this out here to hear other thoughts. I'm good closing this issue if others agree this mitigates the need of the original feature request.

[mholt]

My personal view is that --append-only is best when you are trying to protect yourself from accidental data loss, not from adversaries. Wouldn't the backend/remote credentials be on the computer where the adversary is, anyway? I feel like in the adversarial use case, proxying is the only way to go. I guess we need to know what the OP's situation is.

EDIT: Okay, I could see the value in --append-only still if the backup owner didn't want an adversary, if they took over their computer, to be able to delete their remote backups. In the proxy situation, the adversary would not have credentials to the cloud storage, but even with proxying without --append-only, they could still run delete commands that get proxied, I suppose.

[MatthewVance]

@mholt I tend to agree. --append-only is most helpful for protection against data loss, but you're edited comment is where I ultimately land. Ideally, --append-only and proxying would be used together to cover both concerns.

For the super paranoid, the backend storage could also be configured to to retain deleted objects for a period of time:

• https://docs.aws.amazon.com/AmazonS3/latest/dev/Versioning.html
• https://azure.microsoft.com/en-us/blog/soft-delete-for-azure-storage-blobs-now-in-public-preview/
• https://cloud.google.com/storage/docs/object-versioning

[fd0]

So, I dunno, in my mind, since credentials don't have to be given to the machine doing the backup, does that eliminate the need for the original feature request (--append-only in the caddy plugin)? If so, we can close this.

No, that's not the case, it's a slightly different case. Suppose you have a machine (e.g. a server) which does backups of important data with restic to B2, and attackers gained access to this server.

Up to now, the attackers can just extract the credentials for B2 and delete all data in the bucket. After all, restic needs these credentials for its operation,so they must be present on the server.

When you use rclone as a proxy, this is not the case any more: The attackers would just find access credentials for the rclone (e.g. an SSH key, or username and password for the REST backend). So they can't directly access the data stored at B2. But they are still able to delete all data in the repository, because that's what restic needs to be able to do to run prune.

This is where the append-only mode comes into play: The idea here is that you backup to rclone or the REST server, and it restricts the client to only add data (except lock files). It isn't possible to run prune any more from this machine, but on the upside attackers cannot delete any data, even if they can access rclone or the REST server.

Using rclone as a proxy hides the B2 credentials, the append-only mode disallows attackers to delete data. It also helps against accidental data loss. ;)

Does this description make the use case clearer to you?

[mholt]

Yep! Thanks, that makes sense.

[bricewge]

Nice to learn about rclone integration and the ability to proxy to other backend. I just found out that rclone now also support --append-only rclone/rclone#2186, to me all the security concern I had about data loses are met with thoses two features.

I'll migrate from this plugin to rclone; feel free to close this issue if you think it's not relevant anymore since we have a better integration with rclone.

[fd0]

Ah, thanks a lot for the pointer to the PR for rclone. It helped me find a bug in rclone as well as in the REST server :)

rclone/rclone#2195
rclone/rclone#2196
restic/rest-server#64