Skip to content
This repository has been archived by the owner on Mar 30, 2023. It is now read-only.

Fix RefreshTokenException('Could not find the Guest token in HTML') #1322

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

LinqLover
Copy link
Contributor

About the problem

This problem has recently begun to occur on some environments.
This doesn't happen every time, so if you are lucky, you don't get the error.

The cause is literally that twint could not find the Guest token in HTML.
Actually, sometimes token isn't included in HTML recently.

#!/usr/bin/env python3
# This program is WTFPL.
import requests
res = requests.get('https://twitter.com')
print(res.text.split('\n')[-1])

twint require the result of running the above code is })();</script><script nonce="VALUE">document.cookie = decodeURIComponent("gt=VALUE; Max-Age=VALUE; Domain=.twitter.com; Path=/; Secure");</script>.
However, sometimes the result is only })();</script> and missing the
Guest token.

About the solution

In this patch, twint get the Guest token from https://api.twitter.com/1.1/guest/activate.json if could not find the one.
The author referred to the code of gallery-dl.

Author's note:

I don't understand session of requests, so the code may be not good.
I hope someone rewrite the patch better and create a pull request.

This commit was adopted from #1320 (comment) by @minamotorin.

Closes #1320.

ABOUT THE PROBLEM

This problem has recently begun to occur on some environments.
This doesn't happen every time, so if you are lucky, you don't get the
error.

The cause is literally literally that twint could not find the Guest
token in HTML.
Actually, sometimes token isn't included in HTML recently.

    #!/usr/bin/env python3
    # This program is WTFPL.
    import requests

res = requests.get('https://twitter.com')
print(res.text.split('\n')[-1])
twint require the result of running the above code is })();</script><script nonce="VALUE">document.cookie = decodeURIComponent("gt=VALUE; Max-Age=VALUE; Domain=.twitter.com; Path=/; Secure");</script>.
However, sometimes the result is only })();</script> and missing the
Guest token.

ABOUT THE SOLUTION

In this patch, twint get the Guest token from
https://api.twitter.com/1.1/guest/activate.json if could not find the
one.
The author referred to the code of gallery-dl:
https://github.com/mikf/gallery-dl/blob/47eae4c393f09937a5dbcc2cb978702fb173e747/gallery_dl/extractor/twitter.py#L780-L783

Author's note:

> I don't understand session of requests, so the code may be not good.
> I hope someone rewrite the patch better and create a pull request.

This commit was adopted from:
twintproject#1320 (comment)

Closes twintproject#1320.
@rangersmyth74
Copy link

Hey all..

I am confused a little.

Q1: Do I add this code to a file?
Q2: Which file please!

#!/usr/bin/env python3

This program is WTFPL.

import requests
res = requests.get('https://twitter.com')
print(res.text.split('\n')[-1])

OR...

Q3: Do I have to download or use this >>> https://api.twitter.com/1.1/guest/activate.json
Q4: The author referred to the code of gallery-dl. > Do I need this code also?

Thanks in advance!

@LinqLover
Copy link
Contributor Author

All you need to do is to replace the token.py in your installation of the twint package. To find out where you have the package installed, simply open a python interpreter in your shell, import twint, and then print twint.__file__. Navigate to this directory and replace the contents of token.py with the contents from the file in this branch.

Alternatively, you can reinstall twint from this location:

pip3 install git+https://github.com/Museum-Barberini/twint.git@fix/RefreshTokenException#egg=twint

@minamotorin
Copy link

After reinstalling via pip3 install git+https://github.com/Museum-Barberini/twint.git@fix/RefreshTokenException#egg=twint, don't forget this patch.

minamotorin added a commit to minamotorin/twint that referenced this pull request Feb 6, 2022
Reference: twintproject#1328, twintproject#1322

This problem doesn't happen recently, but too big is better than too small.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.