S3 touch fails on files greater than 5GB in size

[qodfathr]

Error: buildx failed with: ERROR: failed to solve: failed to touch file: operation error S3: CopyObject,
https response error StatusCode: 400, RequestID: T5TBZT4GG16YVFWR, HostID: /E7Ab6HtmB4mirRGrbve8dmVvEZqnyRrO19i/eWacbsFguCAsdXwYTJ1Q0Hm7413DcoKGZE29X8=,
api error InvalidRequest: The specified copy source is larger than the maximum allowable size for a copy source: 5368709120

I suspect the problem lies here:

buildkit/cache/remotecache/s3/s3.go

Lines 445 to 458 in d7f7786

	func (s3Client *s3Client) touch(ctx context.Context, key string) error {
	copySource := fmt.Sprintf("%s/%s", s3Client.bucket, key)
	cp := &s3.CopyObjectInput{
	Bucket: &s3Client.bucket,
	CopySource: &copySource,
	Key: &key,
	Metadata: map[string]string{"updated-at": time.Now().String()},
	MetadataDirective: "REPLACE",
	}
	_, err := s3Client.CopyObject(ctx, cp)
	return err
	}

and can be corrected by making use of multi-part uploads, or, alternatively, it's my understanding that renaming a file in S3 to have the exact same name as it currently has will effectively 'touch' the file while avoiding the overhead of copy.

[tonistiigi]

(slight offtopic) yikes, does https://github.com/moby/buildkit/blob/v0.13.2/cache/remotecache/s3/s3.go#L213-L219 read the layer into memory before uploading?

and can be corrected by making use of multi-part uploads, o

Seems like this is supported by S3 client, but different methods need to be called. If you understand this logic, feel free to propose a PR.

[qodfathr]

Thanks for the response, @tonistiigi !

(slight offtopic) yikes, does https://github.com/moby/buildkit/blob/v0.13.2/cache/remotecache/s3/s3.go#L213-L219 read the layer into memory before uploading?

It seems like it does...

Seems like this is supported by S3 client, but different methods need to be called. If you understand this logic, feel free to propose a PR.

I have a fix that's not super-efficient (it basically does a multi-part copy when the file size is >5GB), but it does follow the pattern of the existing code. I'm doing some additional testing on it, but so far it has solved my problem. I'll get a PR ready.

[tonistiigi]

It seems like it does...

Fix for that landed in #4551