Skip to content

ChenglongMa/skipper-gcstorage

Repository files navigation

skipper emblem - face of a ship's captain Google Cloud Storage Blob Adapter

npm GitHub release (latest by date) NPM

Google Cloud Storage adapter for receiving upstreams. Particularly useful for handling streaming multipart file uploads from the Skipper body parser.

Installation

Option 1: NPM Package

NPM

$ npm i skipper-gcstorage

Option2: GitHub Package [Deprecated: GitHub package won't update any more, the last version is v2.2.0]

GitHub

$ npm i @chenglongma/skipper-gcstorage

Changelog

Ver 2.3.0

  1. Add keepName option which can set whether to use original filename or a generated UUID name.
  2. Update dependencies to the latest version.

Ver 2.2.0

  1. Update dependencies to the latest version.
    1. sharp: 0.29.2, now it supports M1 chipset ( thanks lahiruelectrily (github.com))
    2. mime: 3.0.0

Ver 2.1.0

Thanks jspark-gigworks (Anselmo Park) so much for his comments!

  1. Emit writefile event when finishing the job.
  2. Support additional CreateWriteStreamOptions listed in https://googleapis.dev/nodejs/storage/latest/global.html#CreateWriteStreamOptions.

Ver 2.0.0

  1. Add resize options, which can compress the images before uploading.

Usage

req.file('avatar')
    .upload({
        // Required
        adapter: require('skipper-gcstorage'),
        bucket: 'existing_or_new_bucket_name', // Will create new one if no such bucket exists.
        // Optional
        projectId: 'GOOGLE_CLOUD_PROJECT', // Mandatory if `keyFilename` was specified.
        keyFilename: '/path/to/GOOGLE_APPLICATION_CREDENTIALS.json',
        bucketMetadata: {
            location: 'us-west1',
        },  // Refer to https://googleapis.dev/nodejs/storage/latest/global.html#CreateBucketRequest
        maxBytes: 60000,
        metadata: {},
        public: true,
        gzip: true,
        keepName: false,
        // Other options in `CreateWriteStreamOptions`
        // Refer to https://googleapis.dev/nodejs/storage/latest/global.html#CreateWriteStreamOptions
        ...CreateWriteStreamOptions,
        resize: {
            width: 500,
            height: 500
        }, // Refer to https://sharp.pixelplumbing.com/api-resize#resize
    }, function whenDone(err, uploadedFiles) {
        if (err) {
            return res.serverError(err);
        }
        return res.ok({
            files: uploadedFiles,
            textParams: req.params.all()
        });
    });

Please don't check in your GCP credentials :)

Parameters

Name Type Required Default Value Description
bucket string Yes None Bucket name in GCP, will create new one if there is no such bucket.
projectId string Yes if keyFilename is specified None, will try to read process.env.GOOGLE_CLOUD_PROJECT if not set "GOOGLE_CLOUD_PROJECT", please refer to Google Cloud Storage#using-the-client-library
keyFilename string No None, will try to read process.env.GOOGLE_APPLICATION_CREDENTIALS if not set "/path/to/GOOGLE_APPLICATION_CREDENTIALS.json"
bucketMetadata dictionary No {} Metadata to set for the bucket. Refer to Google Cloud Storage#CreateBucketRequest
metadata dictionary No {} Extra info attached to the file
public boolean No true Whether to make the file public
keepName boolean or string No false Whether to use original filename. The uploaded file will be set to:
* a UUID name if keepName=false;
* its original name if keepName=true;
* the value of keepName if keepName is a string
...CreateWriteStreamOptions expanded dictionary No {} Options for File#createWriteStream(). Refer to Google Cloud Storage#CreateWriteStreamOptions
resize dictionary with keys width and height No {} The new size of image. Only works when the file is an image. Refer to sharp#resize.

NOTE

  1. Skipper-GCStorage will create new bucket if specified one does not exist.

    1. Assign bucket metadata into bucketMetadata.
  2. Support multiple ways for Authentication

    1. Specify projectId AND keyFilename;
    2. export GOOGLE_APPLICATION_CREDENTIALS environment variable;
    3. Login with an eligible service account;
    4. *For more details, please refer to https://cloud.google.com/docs/authentication/production#command-line.
  3. Use with sails-hook-uploads for better results :)

  4. Cloud Storage operates with a flat namespace, which means that folders don't actually exist within Cloud Storage. If you create an object named folder1/file.txt in the bucket your-bucket, the path to the object is your-bucket/folder1/file.txt, but there is no folder named folder1; instead, the string folder1 is part of the object's name.

    Please refer to https://cloud.google.com/storage/docs/folders

Acknowledgement

  1. Sails Skipper
  2. Skipper-S3
  3. jspark-gigworks (Anselmo Park)

Contribution

Pull requests are welcome!