You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
If you're doing a copy from one region to another with --no-clobber there are a lot of failure modes with region resolution. The resolution order of ls is
Region detection
While executing the commands, s5cmd detects the region according to the following order of priority:
--source-region or --destination-region flags of cp command.
AWS_REGION environment variable.
Region section of AWS profile.
Auto detection from bucket region (via HeadBucket API call).
us-east-1 as default region.
So this is actually impossible if the region section of your aws profile is set (which you might want as a default). Without the config, it relied on HeadBucket calls, which can sometimes cause performance problems when making a lot of copies with s5cmd run.
Ideally, s5cmd would take in a --region argument, and s5cmd cp $SRC $DST --destination-region $REGION --no-clobber would pass the --destination-region flag to the internal list needed by --no-clobber.
The text was updated successfully, but these errors were encountered:
If you're doing a copy from one region to another with
--no-clobber
there are a lot of failure modes with region resolution. The resolution order of ls isSo this is actually impossible if the region section of your aws profile is set (which you might want as a default). Without the config, it relied on HeadBucket calls, which can sometimes cause performance problems when making a lot of copies with
s5cmd run
.Ideally, s5cmd would take in a
--region
argument, ands5cmd cp $SRC $DST --destination-region $REGION --no-clobber
would pass the--destination-region
flag to the internal list needed by--no-clobber
.The text was updated successfully, but these errors were encountered: