zs3server - a no-code decentralized storage server

zs3server provides a no-code s3-compatible decentralized storage server on Züs allocation using a minio-gateway interface.

• Züs Overview
• Architecture
• Building zs3 server
• Running zs3 server
• Test MiniO client
• Configure Minio client

Züs Overview

Züs is a high-performance cloud on a fast blockchain offering privacy and configurable uptime. It is an alternative to traditional cloud S3 and has shown better performance on a test network due to its parallel data architecture. The technology uses erasure code to distribute the data between data and parity servers. Züs storage is configurable to provide flexibility for IT managers to design for desired security and uptime, and can design a hybrid or a multi-cloud architecture with a few clicks using Blimp's workflow, and can change redundancy and providers on the fly.

For instance, the user can start with 10 data and 5 parity providers and select where they are located globally, and later decide to add a provider on-the-fly to increase resilience, performance, or switch to a lower cost provider.

Users can also add their own servers to the network to operate in a hybrid cloud architecture. Such flexibility allows the user to improve their regulatory, content distribution, and security requirements with a true multi-cloud architecture. Users can also construct a private cloud with all of their own servers rented across the globe to have a better content distribution, highly available network, higher performance, and lower cost.

The QoS protocol is time-based where the blockchain challenges a provider on a file that the provider must respond within a certain time based on its size to pass. This forces the provider to have a good server and data center performance to earn rewards and income.

The privacy protocol from Züs is unique where a user can easily share their encrypted data with their business partners, friends, and family through a proxy key sharing protocol, where the key is given to the providers, and they re-encrypt the data using the proxy key so that only the recipient can decrypt it with their private key.

Züs has ecosystem apps to encourage traditional storage consumption such as Blimp, a S3 server and cloud migration platform, and Vult, a personal cloud app to store encrypted data and share privately with friends and family, and Chalk, a zero upfront cost permanent storage solution for NFT artists.

Other apps are Bolt, a wallet that is very secure with air-gapped 2FA split-key protocol to prevent hacks from compromising your digital assets, and it enables you to stake and earn from the storage providers; Atlus, a blockchain explorer and Chimney, which allows anyone to join the network and earn using their server or by just renting one, with no prior knowledge required.

Architecture

There are three main components that will be installed in the customer server.

1. ZS3Server is the main component that will communicate directly with Züs storage.

2. LogSearch API is the log component that will store the audit log from the S3 server and it will be consumed using ZS3 API

3. MinioClient is the component that will communicate directly to the zs3server and it is protected using access and secret key.

Building zs3-server

Prerequisites to run MinIO ZCN gateway:

• A wallet.json created using zwalletcli
• Config.yaml
• An allocation.txt created using zboxcli.

git clone git@github.com:0chain/zs3server.git
cd zs3server
go mod tidy
go build .
export MINIO_ROOT_USER=someminiouser
export MINIO_ROOT_PASSWORD=someminiopassword
./minio gateway zcn --configDir /path/to/config/dir
Note: allocation and configDir both are optional. By default configDir takes ~/.zcn as configDir and if allocation is not provided in command then it will look for allocation.txt file in configDir directory.

If you want to debug on local you might want to build with -gcflags="all=-N -l" flag to view all the objects during debugging.

Running zs3-server

1. To build and run the minio server component you need to install docker.

2. Run the docker-compose command inside the zs3server directory./

docker-compose -f environment/docker-compose.yaml up -d

3. Make sure allocation.txt file exist in the default folder ~/.zcn

4. Now you can interact with the clint API follow this doc

5. You can also interact with the logsearch API by following this doc

Test using AWS Client aws

Installation

Install from here: https://aws.amazon.com/cli/

Configuration

Fetch the access key and secret from your deployed zs3server. To configure aws cli, type aws configure and specify the zs3server key information like below:

aws configure
AWS Access Key ID [None]: miniouser
AWS Secret Access Key [None]: miniopassword
Default region name [None]: us-east-1
Default output format [None]: ENTER

Additionally enable AWS Signature Version ‘4’ for zs3server.

aws configure set default.s3.signature_version s3v4

Examples

To list your buckets

aws --endpoint-url https://localhost:9000 s3 ls
2016-03-27 02:06:30 deebucket
2016-03-28 21:53:49 guestbucket
2016-03-29 13:34:34 mbtest
2016-03-26 22:01:36 mybucket
2016-03-26 15:37:02 testbucket

To list contents inside bucket

aws --endpoint-url https://localhost:9000 s3 ls s3://mybucket
2016-03-30 00:26:53      69297 argparse-1.2.1.tar.gz
2016-03-30 00:35:37      67250 simplejson-3.3.0.tar.gz

To make a bucket

aws --endpoint-url https://localhost:9000 s3 mb s3://mybucket
make_bucket: s3://mybucket/

To add an object to a bucket

aws --endpoint-url https://localhost:9000 s3 cp simplejson-3.3.0.tar.gz s3://mybucket
upload: ./simplejson-3.3.0.tar.gz to s3://mybucket/simplejson-3.3.0.tar.gz

To delete an object from a bucket

aws --endpoint-url https://localhost:9000 s3 rm s3://mybucket/argparse-1.2.1.tar.gz
delete: s3://mybucket/argparse-1.2.1.tar.gz

To remove a bucket

aws --endpoint-url https://localhost:9000 s3 rb s3://mybucket
remove_bucket: s3://mybucket/

Test using MinIO Client mc

mc provides a modern alternative to UNIX commands such as ls, cat, cp, mirror, diff etc. It supports filesystems and Amazon S3 compatible cloud storage services.

Installation

Install from here for your os: https://min.io/docs/minio/macos/index.html

Configure MinIO Client

mc config host add zcn http://localhost:9000 miniouser miniopassword

Examples

To list your buckets

mc ls zcn/
2016-03-27 02:06:30 deebucket
2016-03-28 21:53:49 guestbucket
2016-03-29 13:34:34 mbtest
2016-03-26 22:01:36 mybucket
2016-03-26 15:37:02 testbucket

To list contents inside bucket

mc ls zcn/mybucket
2016-03-30 00:26:53      69297 argparse-1.2.1.tar.gz
2016-03-30 00:35:37      67250 simplejson-3.3.0.tar.gz

To make a bucket

mc mb zcn/mybucket
make_bucket: zcn/mybucket

To add an object to a bucket

mc cp simplejson-3.3.0.tar.gz zcn/mybucket
upload: ./simplejson-3.3.0.tar.gz to zcn/mybucket/simplejson-3.3.0.tar.gz

To delete an object from a bucket

mc rm zcn/mybucket/argparse-1.2.1.tar.gz
delete: zcn/mybucket/argparse-1.2.1.tar.gz

To remove a bucket

mc rb zcn/mybucket
remove_bucket: zcn/mybucket/

Check mc --help for the exhaustive list of cmds available.

Test using Postman Using REST APIs

• Add the following authorization settings
• The AccessKey would be the MINIO_ROOT_USER which you set earlier during zs3server deployment and SecretKey would be the MINIO_ROOT_PASSWORD.
• If you do not want to share the MINIO_ROOT_USER and MINIO_ROOT_PASSWORD, you can also create a user from minio console and share their access key and secret instead.
• Use the REST APIs to interact with the server.
• Postman collection for the same is provided below: Postman Collection