Large channels triggering rate limits on conversations.info due to many SlackRoom instances created

[markusj]

I tried to migrate an errbot instance running a bot plugin from the old errbot slack backend to the SlackV3 backend. However, the instance pretty quickly hits a wall:

slack_sdk.errors.SlackApiError: The request to the Slack API failed. (url: https://www.slack.com/api/conversations.info)
The server responded with: {'ok': False, 'error': 'ratelimited'}

This happens while the backend calls conversations_info(). conversations.info is a Tier-3 method. Hence, the rate limit is about 50 calls/minute. However, conversations_info() is called each time a SlackRoom is created (via _cache_channel_info()). This happens for each SlackRoomOccupant that is created, e.g. when accessing SlackRoom.occupants(). Thus, the rate limit is easily hit in bigger channels. To address this issue, some kind of caching (#3) is required to prevent calling conversations.info when creating SlackRoom instances. Especially for occupants(), since the SlackRoom of each of the occupants is self and creating another instance is avoidable?

Full trace

Traceback (most recent call last):
File "/home/mjsvc/.virtualenvs/hserrbot/lib/python3.6/site-packages/errbot/plugin_manager.py", line 521, in activate_plugin
self._activate_plugin(plugin, plugin_info)
File "/home/mjsvc/.virtualenvs/hserrbot/lib/python3.6/site-packages/errbot/plugin_manager.py", line 474, in _activate_plugin
plugin.activate()
File "/home/mjsvc/etc/hserrbot/data/plugins/markusj/titlebot-ng/titlebot.py", line 1170, in activate
self.tryAddRoom(room)
File "/home/mjsvc/etc/hserrbot/data/plugins/markusj/titlebot-ng/titlebot.py", line 1122, in tryAddRoom
occupants = room.occupants # cache occupants
File "/home/mjsvc/local/src/err-backend/err-backend-slackv3/_slack/room.py", line 229, in occupants
SlackRoomOccupant(self._webclient, member, self.id, self._bot)
File "/home/mjsvc/local/src/err-backend/err-backend-slackv3/_slack/room.py", line 277, in init
self._room = SlackRoom(webclient=webclient, channelid=channelid, bot=bot)
File "/home/mjsvc/local/src/err-backend/err-backend-slackv3/_slack/room.py", line 48, in init
self._cache_channel_info(channelid)
File "/home/mjsvc/local/src/err-backend/err-backend-slackv3/_slack/room.py", line 92, in _cache_channel_info
res = self._webclient.conversations_info(channel=channelid)
File "/home/mjsvc/.virtualenvs/hserrbot/lib/python3.6/site-packages/slack_sdk/web/client.py", line 2242, in conversations_info
return self.api_call("conversations.info", http_verb="GET", params=kwargs)
File "/home/mjsvc/.virtualenvs/hserrbot/lib/python3.6/site-packages/slack_sdk/web/base_client.py", line 145, in api_call
return self._sync_send(api_url=api_url, req_args=req_args)
File "/home/mjsvc/.virtualenvs/hserrbot/lib/python3.6/site-packages/slack_sdk/web/base_client.py", line 189, in _sync_send
additional_headers=headers,
File "/home/mjsvc/.virtualenvs/hserrbot/lib/python3.6/site-packages/slack_sdk/web/base_client.py", line 323, in _urllib_api_call
status_code=response["status"],
File "/home/mjsvc/.virtualenvs/hserrbot/lib/python3.6/site-packages/slack_sdk/web/slack_response.py", line 205, in validate
raise e.SlackApiError(message=msg, response=self)
slack_sdk.errors.SlackApiError: The request to the Slack API failed. (url: https://www.slack.com/api/conversations.info)
The server responded with: {'ok': False, 'error': 'ratelimited'}

[nzlosh]

Thank you for reporting this issue. I'll look into what can be done to improve the situation. How many occupants were in the room that caused the rate limit?

[markusj]

The room has about 1100 members 😅