NSFS | Support anonymous NSFS requests #7997

naveenpaul1 · 2024-04-24T15:14:48Z

Explain the changes

Support anonymous NSFS requests
add CLI for add/updat/delete/status the anonymous account
Ceph test test_post_object_wrong_bucket added to black list
load_requesting_account() will load account from anonymous account. UID and GID or distinguished_name from anonymous account use for this.
anonymous user to account_cache with empty access key and for that read_account_by_access_key() method is updated

Issues: Fixed #xxx / Gap #xxx

Changes added to support anonymous request in namespace store
remove namesapce store check in code that prevent anonymous request

Testing Instructions:

Add anonymous account
node src/cmd/manage_nsfs account add --anonymous --uid {uid} --gid {gid}
Create Bucket with policy that support anonymous request
{ Effect: 'Allow', Principal: { AWS: ["*"] }, Action: ['s3:GetObject', 's3:ListBucket'], Resource: ['arn:aws:s3:::*'] }
access object item using anonymous s3 request

Doc added/updated
Tests added

romayalon · 2024-05-09T10:39:09Z

Hi @naveenpaul1 :)
This week I added the test_s3_bucket_policy.js to the Non Containerized tests, I saw some anonymous tests in this tests file and skipped them, can you please check it and unskip the relevant tests?
reference - #8022

src/endpoint/s3/s3_rest.js

src/sdk/object_sdk.js

src/cmd/nsfs.js

src/manage_nsfs/manage_nsfs_validations.js

src/sdk/object_sdk.js

src/server/system_services/schemas/nsfs_account_schema.js

src/manage_nsfs/manage_nsfs_constants.js

src/cmd/manage_nsfs.js

src/manage_nsfs/manage_nsfs_validations.js

docs/non_containerized_NSFS.md

src/manage_nsfs/manage_nsfs_validations.js

src/util/native_fs_utils.js

src/manage_nsfs/manage_nsfs_cli_utils.js

src/cmd/manage_nsfs.js

romayalon · 2024-05-27T08:11:07Z

@naveenpaul1 Thanks for the hard work on this one!!!

Signed-off-by: naveenpaul1 <napaul@redhat.com>

romayalon · 2024-05-27T12:24:58Z

src/manage_nsfs/manage_nsfs_cli_utils.js

+ * (instead of flags) and return its content
+ * @param {object[]} access_keys
+ */
+function is_anonymous(access_keys) {


rename to has_access_keys please

romayalon · 2024-05-27T12:26:05Z

docs/non_containerized_NSFS.md

+
+Anonymous requests are those sent without access and secret key and Noobaa NSFS allow it with supporting bucket policy. Users have to create an anonymous account before accessing the resources. Noobaa uses anonymous account's uid and gid or user(distinguished_name) for accessing bucket storage paths. Noobaa can have one anonymous account. Commands for creating anonumous account are
+```
+node src/cmd/manage_nsfs account add --anonymous --uid {uid} --gid {gid}


can someone create/update a regular account called anonymous? we need to block it if so

romayalon · 2024-05-27T12:36:00Z

src/sdk/object_sdk.js

-            // ANONYMOUS: cannot work without bucket, cannot work on namespace bucket (?)
+            // ANONYMOUS: cannot work without bucket.
+            // Return if the acount is anonymous
+            const is_anon = !(token && token.access_key);


this causing s3 requests with access key empty and secret key non empty to make anon requests instead of failing like happens in AWS

guymguym · 2024-05-27T12:47:20Z

src/sdk/bucketspace_fs.js

@@ -96,8 +96,8 @@ class BucketSpaceFS extends BucketSpaceSimpleFS {

    async read_account_by_access_key({ access_key }) {
        try {
-            if (!access_key) throw new Error('no access key');
-            const iam_path = this._get_access_keys_config_path(access_key);
+            const iam_path = access_key === '' ? this._get_account_config_path(config.NC_ANON_ACCOUNT_NAME) :


Using empty string or even undefined can be accidental and we really don't want to allow that.
Lets do this - define a new const anonymous_access_key = Symbol('anonymous_access_key') in object_sdk.
Then we should use this symbol as the key to the cache, but be explicit about it and not implicit.

guymguym · 2024-05-27T12:48:26Z

src/sdk/object_sdk.js

@@ -207,19 +207,17 @@ class ObjectSDK {
    async load_requesting_account(req) {
        try {
            const token = this.get_auth_token();
-            if (!token) return;
+            const is_anon = !(token && token.access_key);


also here we want to only check !token and not assume empty access key is also anon, right?

guymguym · 2024-05-27T12:49:52Z

src/sdk/object_sdk.js

            this.requesting_account = await account_cache.get_with_cache({
                bucketspace: this._get_bucketspace(),
-                access_key: token.access_key,
+                access_key: is_anon ? '' : token.access_key,


Per the suggestion to use symbol above -

Suggested change

access_key: is_anon ? '' : token.access_key,

access_key: token ? token.access_key : anonymous_access_key,

guymguym · 2024-05-27T12:56:50Z

docs/non_containerized_NSFS.md

+```
+or
+```
+node src/cmd/manage_nsfs account add --anonymous --user {user}


change all calls from manage_nsfs to noobaa-cli -

Suggested change

node src/cmd/manage_nsfs account add --anonymous --user {user}

noobaa-cli account add --anonymous --user {user}

naveenpaul1 requested a review from romayalon April 24, 2024 15:14

pull-request-size bot added the size/L label Apr 24, 2024

naveenpaul1 force-pushed the anonymous_namespace_req branch from bba3686 to 9152079 Compare April 24, 2024 15:48

naveenpaul1 requested a review from guymguym April 24, 2024 15:50

naveenpaul1 marked this pull request as draft April 25, 2024 08:46

guymguym added this to the 5.15.3 milestone Apr 25, 2024

naveenpaul1 force-pushed the anonymous_namespace_req branch from 119f473 to 4a7a0e4 Compare April 25, 2024 10:53

naveenpaul1 marked this pull request as ready for review April 25, 2024 11:21

naveenpaul1 force-pushed the anonymous_namespace_req branch 2 times, most recently from 587d9f6 to 29b78b8 Compare April 28, 2024 11:10

guymguym linked an issue Apr 28, 2024 that may be closed by this pull request

standalone nsfs: process getting crashed when we hit anonymous tests #7519

Open

guymguym modified the milestones: 5.15.3, 5.15.4 May 8, 2024

naveenpaul1 force-pushed the anonymous_namespace_req branch 2 times, most recently from 54c3ad4 to 658345d Compare May 9, 2024 07:49

naveenpaul1 changed the title ~~Support anonymous NSFS requests~~ NSFS | Support anonymous NSFS requests May 9, 2024

guymguym requested changes May 9, 2024

View reviewed changes

src/endpoint/s3/s3_rest.js Outdated Show resolved Hide resolved

src/sdk/object_sdk.js Outdated Show resolved Hide resolved

naveenpaul1 force-pushed the anonymous_namespace_req branch 6 times, most recently from e3bb799 to ee482a7 Compare May 14, 2024 11:17

naveenpaul1 requested a review from guymguym May 14, 2024 12:20

naveenpaul1 force-pushed the anonymous_namespace_req branch 3 times, most recently from e428fd8 to aeb811d Compare May 15, 2024 07:24

romayalon reviewed May 15, 2024

View reviewed changes

src/cmd/nsfs.js Outdated Show resolved Hide resolved

naveenpaul1 force-pushed the anonymous_namespace_req branch from aeb811d to 7abb27e Compare May 16, 2024 06:41

naveenpaul1 force-pushed the anonymous_namespace_req branch 4 times, most recently from 1bd0bfd to 14d1ef3 Compare May 22, 2024 07:48

romayalon reviewed May 22, 2024

View reviewed changes

shirady reviewed May 22, 2024

View reviewed changes

docs/non_containerized_NSFS.md Show resolved Hide resolved

src/manage_nsfs/manage_nsfs_validations.js Outdated Show resolved Hide resolved

src/util/native_fs_utils.js Outdated Show resolved Hide resolved

naveenpaul1 force-pushed the anonymous_namespace_req branch 2 times, most recently from e2a19b4 to 6a85286 Compare May 23, 2024 11:36

naveenpaul1 requested review from romayalon and shirady May 23, 2024 12:09

romayalon reviewed May 23, 2024

View reviewed changes

src/manage_nsfs/manage_nsfs_cli_utils.js Outdated Show resolved Hide resolved

romayalon reviewed May 23, 2024

View reviewed changes

src/cmd/manage_nsfs.js Outdated Show resolved Hide resolved

naveenpaul1 force-pushed the anonymous_namespace_req branch 2 times, most recently from bd3dd7f to 8ee713e Compare May 23, 2024 16:12

naveenpaul1 requested a review from romayalon May 23, 2024 16:13

romayalon reviewed May 23, 2024

View reviewed changes

src/cmd/manage_nsfs.js Outdated Show resolved Hide resolved

src/cmd/manage_nsfs.js Outdated Show resolved Hide resolved

src/cmd/manage_nsfs.js Outdated Show resolved Hide resolved

naveenpaul1 force-pushed the anonymous_namespace_req branch 2 times, most recently from d832341 to f309093 Compare May 27, 2024 06:38

naveenpaul1 requested a review from romayalon May 27, 2024 06:39

romayalon approved these changes May 27, 2024

View reviewed changes

Support anonymous NSFS requests

3a7b275

Signed-off-by: naveenpaul1 <napaul@redhat.com>

naveenpaul1 force-pushed the anonymous_namespace_req branch from f309093 to 3a7b275 Compare May 27, 2024 08:35

romayalon reviewed May 27, 2024

View reviewed changes

guymguym reviewed May 27, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NSFS | Support anonymous NSFS requests #7997

NSFS | Support anonymous NSFS requests #7997

naveenpaul1 commented Apr 24, 2024 •

edited

romayalon commented May 9, 2024

romayalon commented May 27, 2024

romayalon May 27, 2024

romayalon May 27, 2024

romayalon May 27, 2024

guymguym May 27, 2024

guymguym May 27, 2024

guymguym May 27, 2024

guymguym May 27, 2024

	access_key: is_anon ? '' : token.access_key,
	access_key: token ? token.access_key : anonymous_access_key,

	node src/cmd/manage_nsfs account add --anonymous --user {user}
	noobaa-cli account add --anonymous --user {user}

NSFS | Support anonymous NSFS requests #7997

Are you sure you want to change the base?

NSFS | Support anonymous NSFS requests #7997

Conversation

naveenpaul1 commented Apr 24, 2024 • edited

Explain the changes

Issues: Fixed #xxx / Gap #xxx

Testing Instructions:

romayalon commented May 9, 2024

romayalon commented May 27, 2024

romayalon May 27, 2024

Choose a reason for hiding this comment

romayalon May 27, 2024

Choose a reason for hiding this comment

romayalon May 27, 2024

Choose a reason for hiding this comment

guymguym May 27, 2024

Choose a reason for hiding this comment

guymguym May 27, 2024

Choose a reason for hiding this comment

guymguym May 27, 2024

Choose a reason for hiding this comment

guymguym May 27, 2024

Choose a reason for hiding this comment

naveenpaul1 commented Apr 24, 2024 •

edited