add SIP: store renter metadata remotely #2794
Conversation
doc/sips/remote-metadata/README.md
Outdated
| Sectors are encrypted with a stream cipher with random access (i.e. | ||
| which can produce bytes for XOR in the arbitrary location of the | ||
| sequence). One contract corresponds to one sequence (i.e. one | ||
| encryption key). The cipher has large space for its pseudorandom |
There was a problem hiding this comment.
Encrypting all of the data in a contract with the same key causes problems for file-sharing. A contract can contain data from multiple files, so sharing the decryption key for one file would mean sharing the key for all other files.
There was a problem hiding this comment.
I overlooked file sharing.
We should use a separate key for each file. Metadata will be encrypted with a per-host key. The only thing to take into account is that if the file is shared (with the key) and somebody knows the list of IDs of hosts involved in storing the contracts (e.g. hosts themselves), then they can decrypt the previous data from the file. It can be not OK for some users who want to publish the final version of a file but keep preliminary versions secretly.
More issues are added by parity sectors. If all data sectors in a group are from the same file, then parity sectors can be encrypted with the same key. But if data sectors of a group originate from multiple files, we have to make sure we don't leak other files when the user shares just one file involved in the group. One idea is to add parity sectors after encrypting the data, so people who obtain the shared file can only decrypt an encrypted version of data from other files. What do you think?
doc/sips/remote-metadata/README.md
Outdated
| The unused end of a metadata sector cannot be encrypted with the same | ||
| key (i.e. just have the pseudorandom sequence without any XORed data) | ||
| because otherwise, the host can decrypt the subsequent records by | ||
| XORing them with what was in the affected bytes before. To avoid this |
There was a problem hiding this comment.
You may want to look into XTS encryption. It uses a "tweakable block cipher" to enable an interesting property: you can overwrite the sector with new encrypted data without compromising security. The only downside is that it's slower.
|
I think we were going to attempt to follow the Bitcoin/bips format. In which case, this sip should be renamed to Bitcoin/bips example: https://github.com/bitcoin/bips |
Move the proposal to directory sip-0001-remote-metadata. Add the table with SIPs and the link to the SIP to sips/README.md.
starius
force-pushed
the
sip-remote-metadata
branch
from
658e112
to
2d17ddb
Compare
|
@tbenz9 I renamed the directory to sip-0001-remote-metadata. I think short description is useful as a part of directory name, like human readable names of git branches. |
doc/sips/README.md
Outdated
|
|
||
| | Number | Layer | Title | Owner | Status | | ||
| | :-----: | :-----: | :---------------------: | :-------: | :-----: | | ||
| | [1][1] | renter | Store renter metadata remotely | [@starius][starius] | draft | |
There was a problem hiding this comment.
please adjust readme spacing to have aligned columns in text format, so a markdown editor is not required to read the table easily when there are many rows.
|
Took the time to review this in full. A lot of this looks good, though some of the proposals around the file format, and other suggested changes may conflict with other stuff that we are working on. I think the biggest strength of this document is highlighting all of the requirements and blocking factors for doing seed based file and contract recovery. Each of the individual proposed solutions though should probably their own SIP, as I think we could go back and forth with each of them a bunch of times, and I wouldn't want those conversations to dilute this document. |
No description provided.