Limit sr3 declare's scope when options and configs specified

[andreleblanc11]

Situation

When using sr3 declare, it's been noticed on multiple occasions that it can be dangerous in a production environment, even when a configuration is specified and limited to the --users option.

Essentially, running sr3 --users declare my-component/my-config will go through all of admin.conf and try to declare all users and all exchanges specified in the configuration.

# How to reproduce
## The new user/exchange is MSC-CA-TEST/xs_MSC-CA-TEST respectively.
## NOTE: my-component/my-config holds the declaration of `source MSC-CA-TEST` and `exchange xs_MSC-CA-TEST`
## NOTE2: `admin.conf` also holds `declare source MSC-CA-TEST` and `declare exchange xs_MSC-CA-TEST`
sr3 --users declare my-component/my-config
...
2024-04-03 13:27:08,769 65011 [INFO] sarracenia.rabbitmq_admin add_user permission user 'Bulletins@ddsr-dev.cmc.ec.gc.ca' role source  configure=....
2024-04-03 13:27:09,672 65011 [INFO] sarracenia.rabbitmq_admin add_user permission user 'MSC-CA-TEST@ddsr-dev.cmc.ec.gc.ca' role source  configure=...
2024-04-03 13:27:09,218 65011 [INFO] sarracenia.rabbitmq_admin add_user permission user 'EUMETSAT@ddsr-dev.cmc.ec.gc.ca' role source  configure=...
...
2024-04-03 13:27:10,091 65011 [INFO] sarracenia.moth.amqp putSetup exchange declared: xs_Bulletins (as: amqps://root@ddsr-dev.cmc.ec.gc.ca/)
2024-04-03 13:27:10,092 65011 [INFO] sarracenia.moth.amqp putSetup exchange declared: xs_EUMETSAT (as: amqps://root@ddsr-dev.cmc.ec.gc.ca/)
2024-04-03 13:27:10,093 65011 [INFO] sarracenia.moth.amqp putSetup exchange declared: xs_MSC-CA-TEST (as: amqps://root@ddsr-dev.cmc.ec.gc.ca/)

Suggestions/Improvements

• When specifying --users only the declare source entries should be affected from admin.conf.
• Have a new separate --exchanges option that would only declare the exchanges from admin.conf
• When specifying a config, limit the user/exchange declaration to only that config and not the whole of admin.conf.

[petersilva]

Why is it marked "invalid" ? I get the gist of the question... not clear how to implement... I saw your system times when you had --users in the declare... yeah... it doesn't care which configs are specified, it checks all users all the time.

• the admin.conf and default.conf are always parsed, so sr3 --users declare would apply to all users declared in either of those.
• those two files are where all the users are declared. so it seems like they would all be declared all the time.

[petersilva]

perhaps the "invalid" tag needs some clarification. it means the issue itself is mis-reported, or inaccurate. it means the bug report isn't any good. that's what "invalid" usually means in issue trackers. I think the submitter is thinking the behavior of the app is not valid... If you mean the behaviour of the app is bad, the correct label is "bug"

[andreleblanc11]

Oh.. I interpreted it as "something isn't right" in the code. My bad

[andreleblanc11]

The way sr3 declare is working right now makes it scary for operational use. Analysts are scared to use it based on the output of the logs mainly.

We shouldn't need to re-declare users/exchanges when it isn't necessary.

[petersilva]

when parsing a config file:

• every file implicitly includes default.conf
• every file implicitly includes admin.conf
• all users should be defined in admin.conf (declare subscribe or declare source) and credentials.conf. NOT in individual configuration files, in order for the admin user to see them.

So all users are defined in files that are included in every config file, it's hard to see what scope an analyst is expecting.

If you don't do all the users all the time, then you have to allow for CRUD (create update delete)...
you inspect the output of some (rabbitmqadmin? query function... then you need to check if the password has changed? has the role changed?

so... how to indicate scope... I guess check the broker and post_broker options, and only deal with users in those settings?

[petersilva]

and it's even an English word... https://en.wiktionary.org/wiki/grappe#:~:text=(file)-,Noun,bunch%2C%20cluster although a heck of a lot more obvious in French.

Actually, I thought the wictionary was talking about English words, but it might just be documenting a French word in English... why it would do that... no idea.

[andreleblanc11]

When running sr3 declare the users get declared via the following and iterate through all users and brokers.

sarracenia/sarracenia/sr.py

Lines 1353 to 1360 in 7d8e901

	if u_url.username in self.default_cfg.declared_users:
	#print( 'u_url : user:%s, pw:%s, role: %s netloc: %s, host:%s' % \
	# (u_url.username, u_url.password, self.default_cfg.declared_users[u_url.username],
	# u_url.netloc, u_url.hostname ))
	sarracenia.rabbitmq_admin.add_user( \
	self.brokers[h]['admin'].url, \
	self.default_cfg.declared_users[u_url.username],
	u_url.username, u_url.password, self.options.dry_run )

For the rest of the code, it iterates through

sarracenia/sarracenia/sr.py

Line 1377 in 7d8e901

for f in self.filtered_configurations:

We already have methods in rabbitmq_admin.py like get_exchanges and get_users that we could use to filter out the already existing stuff.

[petersilva]

discussing... we agree that:

• if no configurations are specified, then work on all of them.
• if some configurations are specified then only create users corresponding to the brokers referred to in those configurations:
  • broker, post_broker, report_broker, ....any other broker setting.

[andreleblanc11]

Plan of action:

• Get a list of users to declare based on filtered_configurations. Compare with all configurations and see if filtering is needed.
• In the source users declaring loop, have a conditional statement to check if the user is in the list. If it is, declare it. If not, it's been filtered out so don't

[andreleblanc11]

To test, on dev

• Break a users / some users permissions.
• Run sr3 declare component/my-config , see if it fixes the permissions
• Run sr3 declare components/my-configs , see if it fixes them
• Run sr3 declare, see if anything changes at all once all permissions back to normal.

In flow test,

• Change permissions before test to match all use cases.

[petersilva]

• In the source declaring loop, have a conditional statement to check if the user is in the list. If it is, declare it. If not, it's been filtered out so don't

It's a user declaring loop... can be either source or subscriber... if that' isn't how it is, then there must be two loops...