Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Twscrapes requests returned different tweets #162

Open
OkFranco opened this issue Apr 7, 2024 · 1 comment
Open

Twscrapes requests returned different tweets #162

OkFranco opened this issue Apr 7, 2024 · 1 comment

Comments

@OkFranco
Copy link

OkFranco commented Apr 7, 2024

Hello, first thanks for the great work.

Recently, I tried to make a search request to the API (api.search(q)) with q= 'ambiente sustentabilidade lang:pt since:2007-01-01 until:2022-12-31 ' (request#1). Afterwards I did the same request but from 2022-01-01 until 2022-12-31 (request #2).

Considering this context I got 2 issues:
1- The API returned some tweets with lang different from pt. This was not a big deal as during data treatment I can overcome this situation. Just an FYI:
2- The tweets returned in the request#2 are not all in the request#1. In addition, I got less tweets from request#1 than in request#2, which is quite strange. Considering this situation, I am having concerns if the API is indeed returning all tweets that validate the query (q) provided.

Thanks for the help!

@atesahmet0
Copy link

Sometimes I have got tweets from different accounts when using a dork:"@from:username". It was like %1 of the tweets or so. I believe it is something related to Twitter.
To overcome second issue you may fetch tweets by 1-2 days interval and fetch more than once to make sure you've got all the tweets. And make sure to set limit to 1000 or more than enough. If your result doesn't have more tweets than limit you should be fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants