Storage
In 0Chain’s network, storage is provided by specialized entities called blobbers.
Blobber: A blobber is responsible for storing data in exchange for rewards.
Our design relies on the use of signed markers, as described in 0Chain Payment architecture page under Token Pools and Markers header. Critically for storage, when the blobber redeems these markers on the blockchain, they also serve as a public commitment to store the data provided. Our protocol was first outlined in a DAPPCON paper [7], as well as a related technical report [2].
For storage, there are different amounts of storage that must be understood for a blobber:
-
capacity: The total storage that a blobber physically offers.
-
staked capacity: The amount of capacity that is also backed by staked tokens, either from the blobber or delegates.
-
free capacity: The amount of staked capacity that has not already been purchased by a client.
-
purchased capacity: Of the staked capacity, the amount that clients have currently purchased, whether they are storing anything or not.
-
used storage: Of the purchased capacity, the amount that the client is currently using.
A stake pool stores tokens backing a specific blobber’s offer of storage. After the offer of storage expires, the stake pool tokens return to the delegates. Note that a stake pool is actually a collection of delegate pools, where each pool represents the tokens belonging to a specific delegate.
A blobber may offer additional capacity at any time. However, capacity can only be lowered if it has not already been staked. Similarly, while a delegate can request to unstake tokens at any time, the request can only be granted when it would not drop the staked capacity below the purchased capacity.
When a blobber registers to provide storage, it specifies its total storage capacity, its pricing for both reads and writes, and the duration (max offer duration)for how long its pricing is valid. The duration starts from the timestamp of the transaction where the offer of storage was first made.
Note that the blobber cannot offer storage immediately. First, there must be enough tokens staked to guarantee service, as discussed in Service Providers, Staking and Delegates . These tokens do not have to be staked by the blobber itself, although we expect that the blobber will provide at least some of the stake. Other clients may serve as delegates, staking tokens on behalf of the blobber and sharing in the rewards offered.
The blobber goes through the same process when it wishes to expand or decrease the storage that it offers, increasing or decreasing the stake amount of tokens needed. A blobber can specify a capacity of 0 if they wish to stop providing storage altogether.
It should be noted that a blobber cannot abandon its existing storage agreements. The blobber must maintain those allocations until the user releases them, or until the duration of the storage offer elapses.
An allocation is a collection of data associated with a client, and may potentially be stored with many blobbers. To set up a new allocation, a client specifies the price range that they are willing to pay for reads and writes, the size of storage they require, and the duration that they need for the storage (specified as an expire parameter).
For each allocation, a client must have two token pools of funds that blobbers can draw on to be rewarded for the work they have done. The pools are:
-
A write pool, used to pay blobbers for any data uploaded and stored.
-
A read pool, used to pay blobbers for any reads of the data stored.
The read pool is associated with client’s wallet so that they can read from any blobbers. The write pool is tied to allocation and its set of specific blobbers. When requesting a new allocation, the client must specify:
-
The price ranges that it is willing to pay for both reads and writes.
-
The size of data that it needs to store.
-
The expiration time for when that storage will no longer be required.
-
The service-level agreement (SLA) parameter challenge completion time.
-
(Optionally) A list of preferred blobbers.
The challenge completion time allows a client to require a certain quality of service from the blobbers. If the blobber has fast storage gear and network,say in a data center it can respond to a challenge faster than a slower one. Of course, a more demanding challenge completion time means that the blobbers will charge a greater premium for their service.
Blobbers and their delegates receive rewards for reads immediately. However,token rewards for writes are instead transferred to a challenge pool. Tokens in this pool are not made immediately available to the blobber or its delegates, but they may receive those rewards after passing a challenge proving that they are storing the data that they claim.
Figure below shows a high level view of the payment process. A client must have funds committed to a write pool before uploading data. Then, when uploading files to the blobber, the client must include write markers along with the files.
Critically, the write markers include the Merkle root hash of the data. When the blobber commits those markers to the blockchain, it serves as the blobbers commitment to the stored data. Redeeming the markers transfers them to the challenge pool. When the blobber is challenged to prove that the data is stored correctly and successfully passes the challenge, the tokens are transferred from the challenge pool to the blobber. See [2] for additional details on the challenge protocol.
A client can change the size or expiration of an allocation. If extending the allocation (by increasing either of these values), the client must negotiate new terms. However, if reducing both of these values, the client may continue to use the existing terms of the allocation.
For a client to extend their allocation, they must have sufficient tokens in their write pool and the blobbers must have sufficient storage capacity. Otherwise the operation will fail.
The client will continue to pay the original rate for their first allocation, but will pay the new rate for the extended period.
When the client reduces its allocation, it may reclaim some of its tokens. However, there is a delay allowing blobbers to still claim the tokens for the services that they have already provided. However, note that any tokens in the challenge pool are not returned to the client; once they leave the write pool, they are considered to have been paid to the blobber and its delegates.
If the client changes the expiration time so that the allocation is already expired, then the allocation is finished and the blobbers may stop storing the client’s data.
For an allocation, If the blobbers fail to provide the agreed storage, the client may cancel it and regain its tokens from the read pool and write pool. Each allocation is allowed a certain number of failed challenges. Once the blobbers have exceeded that number, the client is permitted (but not required) to cancel the allocation. The failed challenges are tallied collectively, so that a single, poor-performing blobber may permit the client to cancel.
[7] Paul Merrill, Thomas H. Austin, Jenil Thakker, Younghee Park, and Justin Rietz. Lock and load: A model for free blockchain transactions through token locking. In IEEE International Conference on Decentralized Applications and Infrastructures (DAPPCON). IEEE, 2019.
[2] Thomas H. Austin, Paul Merrill, Siva Dirisala, and Saswata Basu. 0chain storage protocol, 2018.