S3FS fails quietly on incorrect password

[JeremyWesthead]

Additional Information

Version of s3fs being used (s3fs --version)

V1.90 (commit:unknown)

Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse or dpkg -s fuse)

No package 'fuse' found or similar, but S3FS works fine generally

Kernel information (uname -r)

6.2.0-39-generic

GNU/Linux Distribution, if applicable (cat /etc/os-release)

PRETTY_NAME="Ubuntu 22.04.3 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.3 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

How to run s3fs, if applicable

[X] command line
[] /etc/fstab

s3fs <bucket name> <local empty dir> -o passwd_file=<password file> -o url=<OCI endpoint> -o use_path_request_style -o nonempty

s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs)

Jan 11 13:10:51 yrden s3fs[720477]: s3fs version 1.90(unknown) : s3fs -o passwd_file=<password file> -o url=<OCI endpoint> -o use_path_request_style -o nonempty <bucket name> <local empty dir>
Jan 11 13:10:51 yrden s3fs[720477]: Loaded mime information from /etc/mime.types
Jan 11 13:10:51 yrden s3fs[720482]: init v1.90(commit:unknown) with GnuTLS(gcrypt)
Jan 11 13:10:53 yrden s3fs[720482]: s3fs.cpp:s3fs_check_service(3575): Failed to connect by sigv4, so retry to connect by signature version 2.
Jan 11 13:10:53 yrden s3fs[720482]: s3fs.cpp:s3fs_check_service(3587): Bad Request(host=<OCI endpoint>) - result of checking service.

Details about issue

If the password file (<password file> above) contains a valid but incorrect password, s3fs gives no indication of failure to mount. This has been tested with OCI's buckets - not sure if platform specific.

Cases which cause sensible 'failure to mount' errors (non-0 exitcode and a stderr message):

• Password file has permissions other than 600
• Password file contains whitespace
• Password file is not of correct format

Specific case this issue refers to which has an exitcode of 0 and no stderr message:

• Password file has correct permissions and formatting, but is functionally an incorrect password

Password file contents of access_key:not_real_key should replicate this behaviour. With a valid password file contents, the same command correctly mounts. Such a fail to mount should tell the user that the mount has failed.

[ggtakec]

@JeremyWesthead
I tried it on AWS S3, but in this case I receive a 403 error and s3fs exits with Invalid Credentials.
I checked the code for v1.90 and your error message, and you're probably receiving the 400 error.
This is probably why the Bad Request error is displayed.

Since the error message in your case cannot be parsed, create a PR #2410 and modify it to output the error message included in the response.

[brodly]

I'm finding that s3fs fails silently on many types of errors.

Running this command
mount --target /mnt/newBucket/ --source my-bucket --type fuse.s3fs -o profile=test

will yield these logs if bucket is wrong or creds are wrong:
s3fs.cpp:s3fs_check_service(3593): bucket or key not found(host=https://s3.amazonaws.com) - result of checking service.
s3fs.cpp:s3fs_check_service(3590): invalid credentials(host=https://s3.amazonaws.com) - result of checking service.

s3fs returns exit code 0 and signals to system to mount, in my case these were the logs

systemd[3563169]: mnt-newBucket.mount: Succeeded.
systemd[1]: mnt-newBucket.mount: Succeeded.

not sure if this is intended, seems strange esp for automated mounting solutions. any thoughts?

[ggtakec]

@brodly Thanks for the details.
I was able to confirm that the exit code of the mount command is 0 if it fails to start via the mount command(a case of wrong credential, etc).
(then s3fs has failed to run and cannot be mounted.)

We need to check why the mount command is not inheriting the s3fs exit code.

[brodly]

thanks @ggtakec. I also noticed this behavior when using the s3fs bin directly too. It reported no logs in stdout and just returned a 0. Saw the same logs as I posted above in journalctl and running with a -f.

Since my use case is automated, had to build a little utility to check /proc/mounts to see if we actually mounted the bucket before continuing on.

FYI I have been using v1.89 but was able to also test on 1.90. Havent tested on versions beyond that.

[ggtakec]

Thank you for your reply.
I checked the current master code and confirmed that exit code=0 at that time.
Since the s3fs process is not exiting with 0, we need to know if there is some other way to convey the exit code to the mount command.

[ggtakec]

I looked into this a little more, but the result remains the same that s3fs itself exits with an error code (1), but the mount command always returns 0.
I checked to see if I could manipulate this value using libfuse's functions , but I could not find such I/F exists.
If it fails, there is not no entry in mtab either, so we should check instead of using the result of the mount command as you do now.

If anyone knows the cause of this, I look forward to helping.