Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KeyError: 'timeline' crash on twitter-profile scraper #991

Open
frankiec opened this issue Jun 30, 2023 · 11 comments
Open

KeyError: 'timeline' crash on twitter-profile scraper #991

frankiec opened this issue Jun 30, 2023 · 11 comments
Labels
bug Something isn't working module:twitter upstream

Comments

@frankiec
Copy link

frankiec commented Jun 30, 2023

Describe the bug

when I run:

/bin/snscrape --jsonl --max-results 10 twitter-profile someone

it throws error:

Traceback (most recent call last):
  File "/home/ubuntu/.local/bin/snscrape", line 8, in <module>
    sys.exit(main())
  File "/home/ubuntu/.local/lib/python3.10/site-packages/snscrape/_cli.py", line 323, in main
    for i, item in enumerate(scraper.get_items(), start = 1):
  File "/home/ubuntu/.local/lib/python3.10/site-packages/snscrape/modules/twitter.py", line 1830, in get_items
    instructions = obj['data']['user']['result']['timeline_v2']['timeline']['instructions']
KeyError: 'timeline'

How to reproduce

/bin/snscrape --jsonl --max-results 10 twitter-profile someone

Expected behaviour

It was working until Thu Jun 29 21:35:07 UTC 2023.

Screenshots and recordings

No response

Operating system

ubuntu 16

Python version: output of python3 --version

Python 3.10.6

snscrape version: output of snscrape --version

snscrape 0.6.2.20230321.dev32+gb76f485

Scraper

witter-profile

How are you using snscrape?

CLI (snscrape ... as a command, e.g. in a terminal)

Backtrace

No response

Log output

No response

Dump of locals

No response

Additional context

No response

@frankiec frankiec added the bug Something isn't working label Jun 30, 2023
@0bmay
Copy link

0bmay commented Jun 30, 2023

I have this error too

File "/home/.venv/lib/python3.10/site-packages/snscrape/modules/twitter.py", line 1902, in get_items
instructions = obj['data']['user']['result']['timeline_v2']['timeline']['instructions']
KeyError: 'timeline'

version: v0.7.0.20230622

@JustAnotherArchivist JustAnotherArchivist changed the title twitter-profile CRITICAL snscrape._cli Dumped stack and locals to /tmp/snscrape KeyError: 'timeline' crash on twitter-profile scraper Jun 30, 2023
@JustAnotherArchivist
Copy link
Owner

I'm seeing the same thing on the web interface: the 'Replies' tab on profile pages is empty. This looks like a bug on Twitter's side.

@0bmay
Copy link

0bmay commented Jun 30, 2023

looks like the UserTweets api call works and has data returned, but UserTweetsAndReplies is busted on Twitters end.. call returns 200, but has no data in timeline_v2

@locfinessemonster

This comment was marked as off-topic.

@0bmay

This comment was marked as off-topic.

@Pratham-19

This comment was marked as duplicate.

@locfinessemonster

This comment was marked as off-topic.

@jerrycool123
Copy link

Twitter has blocked every unregistered user from viewing tweets. Is this related?

@0bmay
Copy link

0bmay commented Jun 30, 2023

Twitter has blocked every unregistered user from viewing tweets. Is this related?

not related.. the UserTweets api endpoint is still returning data... UserTweetsAndReplies is still b0rk3d.

@zack-sims413

This comment was marked as duplicate.

@Akhorramrouz

This comment was marked as spam.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working module:twitter upstream
Projects
None yet
Development

No branches or pull requests

8 participants