Cloudflare (e.g. Gitlab login page) 503 / infinite redirect issue

[moorchegue]

Version info:
2.5.0-1

Does the bug happen if you start with --temp-basedir?
Yes.

Description
Gitlab login page returns 503 and infinitely redirects:

Not sure if it's more of a browser or a Gitlab issue. But it works with Firefox.

How to reproduce
:open https://gitlab.com/users/sign_in/

As to what triggers this response in the first place, Gitlab might react to an IP address from China (tried from 2 locations), and to known proxy subnets (tried Fremont, US).

[The-Compiler]

Probably not much qutebrowser can do about this. Can you try the qt6-v2 branch? See #7202 for details.

[alexjp]

I am using qt6-v2 branch and just test it, it worked fine. was able to login properly, in about some 5 tests.

I remember something similar also happening with firedragon (a librewolf fork (a firefox fork)) some months ago.

[rien333]

I'm just now seeing this on the qt6-v2 branch as well, though I'm not 100% sure if gitlab has always been inaccesible.

[angelus2014]

I'm seeing a somewhat different issue, but it's also with the login page. The 'browser check' takes a very long time before it shows the actual login screen (but in my case, it eventually does open the login page). I've tested it also on Chrome and Safari, but there this doesn't happen. Once that check has been done and I log out and log in again, the login screen appears instantly. I'm not sure if I should create a new ticket as the issue does seem to be different from the OP.

[zstg]

GitLab has been inaccessible - the site simply doesn’t me sign in

This bug happens on only one of my machines- which has the rust adblock installed (the other one doesn’t)

Is this a Qt5 bug on Windows 11?

[rien333]

Is this a Qt5 bug on Windows 11

No, there have already been two commenters that 1) use linux, 2) use qt6.

[GasparVardanyan]

My regular qutebrowser instance have the same problem, but when I launch qutebrowser with --qt-arg name scratchqb -B .local/share/scratchqb everything works fine.
I have linked the .local/share/scratchqb/config directory to my regular config (~/.config/qutebrowser) and .local/share/scratchqb/data/greasemonkey to my regular greasemonkey directory.

[GasparVardanyan]

Removing greasemonkey scripts helped me !!

[Lindenk]

I'm also having this issue, although it only started yesterday. I regularly use gitlab so I can confirm it was working previously (I did not update or change any qutebrowser settings recently). This happens on two different systems (arch linux, qutebrowser v2.5.2) and disabling adblock, removing greesemonkey scripts, and/or using a blank config/etc do not seem to work.

[The-Compiler]

@Lindenk I can reproduce. If I'm reading the page right, they turned Cloudflare's "I'm under attack" mode on, and that apparently has a bug where it doesn't let qutebrowser (or other QtWebEngine browsers) do anything at all. Nothing we can do about it I'm afraid.

[johnlane]

I've had this since 16th August, it worked for me on the 15th. I've started a thread on the Gitlab forum https://forum.gitlab.com/t/gitlab-login-cloudflare-503/73759

[pinarruiz]

Same is happening to me, firefox works as usual and qutebrowser does not, I have tried setting up the user agent to be the same as on firefox, but lo luck, I opened an issue (#7346), and it was a duplicate of this one (did not notice).

[dm1tz]

I believe it's actually not a Gitlab specific issue, for me every site that triggers cloudflare protection algorithm results in infinite redirect. (e.g nginx does that too).

[rien333]

@dm1tz I assume that you have to log in to your nginx to trigger the issue?

[pinarruiz]

@dm1tz I assume that you have to log in to your nginx to trigger the issue?

For me it happens just by opening nginx.com:

[The-Compiler]

I think I've seen it on nginx.com myself months ago, but cannot reproduce now.

[The-Compiler]

Another one which has been reported, which does reproduce: https://www.spamhaus.org/

[johnlane]

A workaround is to change the user agent

qutebrowser --temp-basedir ':set content.headers.user_agent "cloudflare 503 workaround"' https://gitlab.com/users/sign-in

tested with gitlab.com and spamhaus.org. I made up the ua string - you can try anything.

[pinarruiz]

A workaround is to change the user agent

qutebrowser --temp-basedir ':set content.headers.user_agent "cloudflare 503 workaround"' https://gitlab.com/users/sign-in

tested with gitlab.com and spamhaus.org. I made up the ua string - you can try anything.

It does work, but setting it up on the config.py does not:

config.set('content.headers.user_agent', 'cloudflare 503 workaround', 'https://gitlab.com/*')

Not even setting it as the following works:

config.set('content.headers.user_agent', 'cloudflare 503 workaround', '*')

Edit: Add more info

[The-Compiler]

@pinarruiz Setting it with a * pattern and without a pattern at all do different things. From the docs:

Note that the value read from JavaScript is always the global value.

(This is due to API limitations in QtWebEngine). Setting it with a pattern of * is not the same as setting it globally.

That being said, setting your UA to something totally different globally will totally break things on other sites, so I can't recommend it.

[pinarruiz]

@pinarruiz Setting it with a * pattern and without a pattern at all do different things. From the docs:

Note that the value read from JavaScript is always the global value.

(This is due to API limitations in QtWebEngine). Setting it with a pattern of * is not the same as setting it globally.

That being said, setting your UA to something totally different globally will totally break things on other sites, so I can't recommend it.

Thanks for the feedback i was not aware of this, however i only setted the value for a moment just to test if it will work and it is only a matter of using it on more websites.

[0xYUANTI]

hm gitlab still redirects for me (have changed user agent and disabled adblock) sometimes I get an error page saying "please enable javascript"? any ideas? (other than switching to sourcehut)

edit: nvm user agent fixed it

[pinarruiz]

hm gitlab still redirects for me (have changed user agent and disabled adblock) sometimes I get an error page saying "please enable javascript"? any ideas? (other than switching to sourcehut)

edit: nvm user agent fixed it

Hi @0xYUANTI,

Could you share the settings that worked for you?

Thanks.

[johnlane]

Just wanted to share.... the workaround of changing the user agent is only necessary for the Gitlab sign-in page. Once signed in it seems to work fine with the default user agent. I found other things (in particular AWS) didn't like the fake user agent.

[mentalisttraceur]

Nothing we can do about it I'm afraid.

We can always rudely yell incessantly into every available CloudFlare customer support channel politely but clearly voice our dissatisfaction to CloudFlare, as customers or potential customers (and we're all potential CloudFlare customers).

For example:

I was considering using your CDN, but I notice your "I am under attack" feature indiscriminately blocks less popular but legitimate web browsers, for example Qutebrowser.

As a {person / business / ...} with {readers / customers / users / ...} I consider this an unacceptable loss to accessibility[, and as a software developer I would never accept such carelessly broad filtering in code review, and it makes me doubt the quality of your product].

So I {won't be / am considering not using} your CDN and DDoS protection products until this is fixed.

[lahwaacz]

the workaround of changing the user agent is only necessary for the Gitlab sign-in page. Once signed in it seems to work fine with the default user agent.

All the Cloudflare behavior seems to be highly geo-location specific. For example, this trick worked for me until yesterday in one location, but stopped working when I moved my laptop to a different location.

[migueldemoura]

Hi @lahwaacz, I'm afraid this is currently not supported, said arg will break the check.

[mschilli87]

@migueldemoura: Any reasons why? Trying to protect oneself against fingerprinting makes one a suspect of attempting to DDOS your client's server? Why should canvas reading be required to login to a source forge or buy train tickets? This is very confusing to me.

[lahwaacz]

I'm sure there are other ways to determine if the incoming connection is made by a human besides taking their fingerprint...

[Soundtoxin]

@migueldemoura I am currently experiencing this issue on GitLab and other sites with similar checks/challenges. I'm clicking the button to export a HAR but it doesn't seem to be doing anything. What is the expected behavior? No file was downloaded and nothing new is on my clipboard.
I was spoofing the UA in the past due to this issue occurring before, but I've unset my UA at the moment and it doesn't seem to work either way anymore.

[deeenes]

Same for me: HAR export does not work, Cloudflare + hCaptcha keeps reloading indefinitely, it already doesn't even show the challenge, just reloads itself every half minute. Then I changed the User-agent to "Stupid Cloudflare". Suddenly no more captcha, the page loaded right away.

[moorchegue]

Same thing happens with any page of the Panasonic website: https://na.panasonic.com/ just keeps cycling back to captcha.

[rien333]

@moorchegue can't reproduce, unfortunately.

[moorchegue]

Same here: https://www.researchgate.net/about

[Dorovich]

Happens here as well: https://4chan.org/

[GasparVardanyan]

Happens here as well: https://4chan.org/

This link works for me.

My QB info: https://paste.the-compiler.org/view/c8c3938b

[moorchegue]

I can confirm that 4chan is affected.

[amano-kenji]

I think there is not much that qutebrowser can do. Cloudflare can block any browser it wants to block.

You better use multiple browsers.

[rien333]

I can confirm that 4chan is affected.

I can't. In fact, I can't reproduce on any of the websites mentioned. To be fair, I did have to complete a cloudfare when I tested 4chan a while back, but there was no infinite loop.

You better use multiple browsers.

That's one solution. @ALL the affected people: are you running the qt6 version of qutebrowser?

[Dorovich]

That's one solution. @ALL the affected people: are you running the qt6 version of qutebrowser?

Nope. Here's my info.

[rien333]

Right. It would be interesting to know if upgrading qtwebengine to the 6.X series would fix the issue, since the 6.X series uses much more recent versions of chrome. My hunch is that it might, since reading the beginning of this thread tells me that the problem relates to the useragent your browser is sending.

Arch linux makes it trivial to install the qt6 version of qutebrowser. For more general instructions/discussion, see #7628 (i.e. run qutebrowser from git, and set set QUTE_QT_WRAPPER=PyQt6)

[evesdropper]

Getting this issue on cloudflare sites - e.g. neovim.io did not work when I went to check documentation (though not with --temp-basedir). Is there any set workaround?

[The-Compiler]

Try using Qt 6, see #7202

[evesdropper]

Ah, I already am using qutebrowser-qt6 from AUR - is that the one?

[The-Compiler]

Yep - make sure :version shows Qt 6 as well. Other than that, nothing that I'm aware of. I'm afraid this still is mostly a Cloudflare issue rather than a qutebrowser issue...

[moorchegue]

Similar thing with Hetzner now. Doesn't look like Cloudflare:

https://accounts.hetzner.com/

[svermeulen]

gitlab.com is having this issue once again. @migueldemoura Is this something cloudflare can fix again on their end?

[amano-kenji]

I think cloudflare wants to hinder non-mainstream web browsers. I mean google could have paid cloudflare.

[migueldemoura]

gitlab.com is having this issue once again. @migueldemoura Is this something cloudflare can fix again on their end?

I'm afraid I can't reproduce the issue with one of the Qt 6 builds as suggested by @The-Compiler:

qutebrowser v2.5.2
Git commit: cd5624f on qt6-v2 (2023-03-17 20:09:26 +0100)
Backend: QtWebEngine 6.3.1, based on Chromium 94.0.4606.126 (from api)
Qt: 6.3.1

Could you try with one of these? Thanks!

[moorchegue]

Gitlab works for me now on:

qutebrowser v2.5.4
Git commit:
Backend: QtWebEngine 5.15.14, based on Chromium 87.0.4280.144
Qt: 5.15.10 (compiled 5.15.8)

[svermeulen]

Yeah sorry all, I didn't read this thread properly. Using qutebrowser Qt 6 instead of Qt 5 fixed it for me

[iambumblehead]

using qutebrowser with qt6 here and unable to login at gitlab because of infinite redirect at the login page. I tried setting various UA strings per the earlier comments in this thread but none of them allowed me to login

[iambumblehead]

this link has more recent discussions at gitlab related to this issue https://gitlab.com/gitlab-org/gitlab/-/issues/421396

I'm able access the signin page when qutebrowser's configuration uses the following. Not sure why the user agent was being set here

c.content.cookies.accept = 'no-3rdparty'
# c.content.headers.user_agent = 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/110.0.0.0 Safari/537.36'