Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to manual toggle between accounts in .db #138

Open
Rouge-Trader opened this issue Mar 7, 2024 · 4 comments
Open

Ability to manual toggle between accounts in .db #138

Rouge-Trader opened this issue Mar 7, 2024 · 4 comments

Comments

@Rouge-Trader
Copy link

I could be missing something, but don't believe there is the ability to chose which account sends a certain request. For my specific functionality I want one of my accounts to read my timeline while a different account is checking HashTags. Seems like it could be a useful feature to add, I'd be happy to contribute if i could get some pointers.
Thanks

@davinkevin
Copy link

+1 with also round-robin to prevent rate limiting

@BonifacioCalindoro
Copy link

BonifacioCalindoro commented Mar 22, 2024

That feature is curcial for debugging accounts and trying to trace if there is a specific limit set in one of them (yes, there are some limits that twscrape is not aware of yet, for example trying to fetch tweet_details from a tweet_id of a shadowbanned account; some accounts fetch the info, some others fetch None, and i believe it's on twitter servers' side so we need a way to debug which account was used each time)

@vladkens
Copy link
Owner

Hi. Here a function get_for_queue in AccountPoll, so you can control which account you want to use, but you need to write SQL query for this.

Example:

import asyncio

from twscrape import API, AccountsPool


class MyPool(AccountsPool):
    def get_for_queue(self, queue: str):
        # for search timeline always use acc1
        if queue == "SearchTimeline":
            return self._get_and_lock(queue, "acc1")

        # for retweeters use acc2 or acc3
        if queue == "Retweeters":
            qs = "SELECT username FROM accounts WHERE username IN ('acc2', 'acc3') ORDER BY RANDOM() LIMIT 1"
            return self._get_and_lock(queue, qs)

        # for all other queries use the default method
        return super().get_for_queue(queue)


async def main():
    poll = MyPool()
    api = API(poll)

    async for tw in api.search("foo", limit=10):
        print(tw)


if __name__ == "__main__":
    asyncio.run(main())

@davinkevin For simple "round robin" possible to use random accounts order (no custom poll requried):

api.poll._order_by = "RANDOM()"

Possible queue names here:
https://github.com/vladkens/twscrape/blob/main/twscrape/api.py#L11

@davinkevin
Copy link

Thank you for the solution, it works like a charm!

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants