cache: support Alibaba Cloud oss backend cache

[njucjc]

Fix #4487

[crazy-max]

Can we have integration tests similar to

buildkit/client/client_test.go

Lines 5224 to 5260 in fc9de56

	func testBasicAzblobCacheImportExport(t *testing.T, sb integration.Sandbox) {
	integration.SkipOnPlatform(t, "windows")
	workers.CheckFeatureCompat(t, sb,
	workers.FeatureCacheExport,
	workers.FeatureCacheImport,
	workers.FeatureCacheBackendAzblob,
	)
	opts := helpers.AzuriteOpts{
	AccountName: "azblobcacheaccount",
	AccountKey: base64.StdEncoding.EncodeToString([]byte("azblobcacheaccountkey")),
	}
	azAddr, cleanup, err := helpers.NewAzuriteServer(t, sb, opts)
	require.NoError(t, err)
	defer cleanup()
	im := CacheOptionsEntry{
	Type: "azblob",
	Attrs: map[string]string{
	"account_url": azAddr,
	"account_name": opts.AccountName,
	"secret_access_key": opts.AccountKey,
	"container": "cachecontainer",
	},
	}
	ex := CacheOptionsEntry{
	Type: "azblob",
	Attrs: map[string]string{
	"account_url": azAddr,
	"account_name": opts.AccountName,
	"secret_access_key": opts.AccountKey,
	"container": "cachecontainer",
	},
	}
	testBasicCacheImportExport(t, sb, []CacheOptionsEntry{im}, []CacheOptionsEntry{ex})
	}

?

I see in the docs that we could use the S3 SDK to access OSS. Can we consider it to reduce dependencies overhead as we support S3 already?: https://www.alibabacloud.com/help/en/oss/developer-reference/use-amazon-s3-sdks-to-access-oss#section-sp0-q7h-dvj

This way we could use the same integration tests as s3 one using minio.

[crazy-max]

Like s3 should we also consider having an env var to set the region? It seems the region is inferred from endpoint url so maybe not needed?: https://www.alibabacloud.com/help/en/oss/developer-reference/go-configure-access-credentials

// Create an OSSClient instance.
// Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. Specify your actual endpoint.

Just asking because I see the SDK having a method to set it: https://github.com/aliyun/aliyun-oss-go-sdk/blob/2bc58741fa507528ff70b501e7e881a92f88ce3e/oss/client.go#L2891-L2895

[njucjc]

Yeah, the region env var is not needed for OSS SDK

[tonistiigi]

per discussion in #4487 I think we need to figure out some external plugin model for this instead as adding lots of builtin code for every possible vendor does not scale.

Isn't using s3 option for this case? There seems to be some compatibility.

[njucjc]

per discussion in #4487 I think we need to figure out some external plugin model for this instead as adding lots of builtin code for every possible vendor does not scale.

Isn't using s3 option for this case? There seems to be some compatibility.

Yes, there are some compatibility between S3 and OSS driver, but in S3 cache code，it does not use concurrent multi-part upload when uploading blobs, so there may be some performance issues if the cached layer is large.

buildkit/cache/remotecache/s3/s3.go

Line 424 in fc9de56

_, err := s3Client.Upload(ctx, input)

I'm not sure if S3 can also implement concurrent multi-part uploads.

I agree we need to figure out some external plugin model for cache driver, I'd like to have a try, but it will be a long term case.

[AkihiroSuda]

I think this should be called "alibaba-oss" to avoid confusion

[AkihiroSuda]

I also wonder if this should be an external plugin that would talk to buildkitd over grpc or something