Forbidden 403 #230
Comments
|
Btw, from the same household and thus the same source IP, I can hit southwest.com and do a test "CHANGE" flight w/ the flight details and it shows the flight itinerary. I am not sure which method your code is using to retrieve the flight details, though. |
|
Oh, and the record locator in question is currently within the 24 hour checkin period for the departure flight.. and was checked in manually yesterday afternoon. The return flight is on this Wednesday afternoon on the same record locator. |
|
This is the new issue that has been come across in #226 and #201. I will consolidate those issues into this one. I can also reproduce this issue in Docker too (both using Chromium and Chrome). I believe this is just a change from #201 that Southwest did that now returns the forbidden error instead of a 429 (which makes more sense). |
|
You probably have already seen this; but in case not, see the Outro section on this post regarding methods of bypassing reCaptcha. The reason I bring this up is we may be dealing with an invisible reCaptcha style algorithm. |
|
@hildebrau Hey. If you dont mind sharing, how did you get the webdriver to work in a Synology NAS? I am getting the following error I am guessing it can't find a web browser anywhere on the NAS. How did you get around that? Thanks |
@Agiang42, I never saw such an issue. Are you running this code as a docker container? If not, that's probably the issue. I have not tried running the python directly on the Synology because I don't believe all the prerequisites could be installed. |
|
I'm seeing this same issue outside of docker, as well. I have two flights booked for the end of March, so if there are any logs that would be helpful or anything I can try with my account between now and then, I'll be happy to do so. |
|
I am also having the same issue running outside docker with develop branch. Ubuntu 22.04 python 3.10.12 Chrome 122.0.6261.69 |
|
Same issue for me as well. Any ideas on a fix? Is there any sense on the logic behind how southwest is determining when to issue the 403? |
I believe this is the same error as #201, just returning a different status code due to Southwest’s improvements they did. That issue’s thread has many details on what I and others have tried, but there is no known fix to this as of yet. Most people are only getting the error in Docker, so you can try to run it with Python instead for now. |
|
Just wanted to give a heads up that I'm running into the |
|
Within Docker, I can still reproduce this issue. However, the requests seem to go through successfully in some parts and fail in others (although they fail at different requests). I've been analyzing the requests that have gone through, used headed mode for the browser within Docker (using Xvfb in SeleniumBase), and messing around with the initialization flags that are being used to start the browser. However, nothing has been working as of yet. Very strange how most people are only seeing it within Docker, but a select few are also seeing it outside of Docker. If this issue starts occurring for me outside of Docker, it will be easier to find out what is wrong but I don't currently have anymore ideas to see what the issue is/how to fix it. |
|
I can repro this 403 issue using a linux server running the python command script directly, and also using docker on the same box. However, everything works on a Mac that is my daily driver. Southwest could possibly be doing some fingerprinting and blocking the connection from untrusted or unseen clients that it assumes to be bots? |
|
This makes sense, as I running raw linux and python and seeing the error. Can we spoof the user-agent maybe? |
|
I just tried this from a Ubuntu 22.04 desktop with Python 3.10.12 and Google Chrome 122.0.6261.111 and the master branch, and it seems to work fine along with the fare check... The system that fails to work is Ubuntu 22.04 server with Python 3.10.12 and Google Chrome 122.0.6261.111 running in an LXC virtual machine same internal network as the machine that works.. If you want logs let me know. |
SeleniumBase already changes the user agent to line up with what headed Chrome has. The user agent can also be changed by adding It seems like a lot of people are running into issues specifically with a server, so I will try on an Ubuntu server VM to see if I can reproduce the issue. |
|
I seem to be encountering a combination of this and #96 on my Unraid box. Running without |
Hey sir! Funny thing, I just setup a droplet/vm (Digital Ocean) to do this, Ubuntu Server 22.04 LTS, getting same error 403, if you are interested I can DM you details how to get into it if you need a setup to troubleshoot that is already built? |
I can reproduce the issue with an Ubuntu headless server too so that definitely widens the scope of the issue and will make it easier for me to debug (and hopefully fix).
@drippyer If this is still an issue after the 403 issue is fixed, feel free to make a separate issue for it.
@j0nsgh I appreciate the offer. I was able to reproduce it on an Ubuntu server, so now I can troubleshoot this further. |
|
@jdholtz Is there a fix for this? Getting 403s on every reservation I put into the config.json. Same error when just using the command line. Same result in docker and not. |
Currently not. It has been working only on my laptop, which has graphics (desktop, window manager, etc.) so if you aren't already running the script on a device that has graphics you can try that--although it isn't the most convenient. You can also try changing line 118 in I'll have more time to work on figuring out this issue after the middle of next week. |
|
FWIW, I was hitting these issues as well. Pulled the latest as of today and ran this through a Docker and it's working. |
|
I've also been monitoring this issue for a fix, but noticed yesterday (without updating versions) that the 403 is gone and was able to retrieve reservations. Perhaps a change was made on Southwest's side. |
I can change it if many people aren’t running into the issue with Chrome. However, my laptop runs Chromium just fine with this script. On the other hand, I have Chrome on an Ubuntu server and am still reaching the 403. |
|
Is there a way to switch with the docker container? |
I have a Dockerfile that uses Chrome that I can send in a few hours. I haven’t been having much success with it, but it would be good to have someone else test it. |
|
This person just posted on a couple of other similar projects. Just passing this along. |
|
@FuzzyMistborn here's the Dockerfile. To build, just run FROM python:3.12-slim
WORKDIR /app
ENV AUTO_SOUTHWEST_CHECK_IN_DOCKER 1
RUN apt-get update && apt-get upgrade
RUN apt-get install -y wget
RUN wget -q https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
RUN apt-get install -y ./google-chrome-stable_current_amd64.deb
COPY requirements.txt requirements.txt
RUN pip3 install --upgrade pip && pip3 install --no-cache-dir -r requirements.txt
COPY . .
ENTRYPOINT ["python3", "-u", "southwest.py"] |
|
Still got a Forbidden 403 using the Chrome dockerfile :-( Edit: Running this via python works just fine as you seem to have also run into. So weird. |
|
Today I restarted the script and now I'm getting 403's again even in Python. So it's not just a Docker thing it seems. |
|
I've done a little more digging and here's what I've found (no solution yet unfortunately): https://mobile.southwest.com/api/mobile-air-booking/v1/mobile-air-booking/feature/shopping-details is the Using valid headers from my laptop on an Ubuntu server makes the 403 issue go away. This indicates that the issue comes directly from the browser incorrectly generating headers (meaning changing anything with the way the requests module is used will not affect it). I've also inspected the browser window object that the script's browser formulates on my laptop (which does not hit a 403) and a server (which receives a 403) but there doesn't appear to be any differences between them. Additionally, the original request headers (user agent, content-type, etc.--not the cryptic headers generated by the JS) are the same which indicates Southwest isn't detecting the browser with a simple user agent check. I'm inclined to think that some sort of display is "needed" which isn't installed on servers (and causes the browser to generate invalid headers). This would also explain why it is working for people on desktops and laptops (although some have reported it hitting a 403 still which contradicts this. I'm not entirely sure if they were running on a server or headed computer though). |
Do you have that JS code to generate the headers? |
You can find it in a series of requests made when you load https://mobile.southwest.com/check-in (in the network tab of your browser). People have actually tried to reverse engineer the API before (see here), but their engineers appear to have done a very good job to only have it run in a browser environment (hence, why this project needs to use a browser to generate the headers). |
|
For the record, I had the script running in a Docker container using username/password. It successfully found the two flights in the round trip and checked in for the first one. Then when it went to do the second flight days later, it got the 403. So I assume the error can't be because of the environment I was running it in. |
I've been getting similar results, where about 25% of the time I don't get a 403 and it schedules my flights fine. A possible explanation is that they are using AI through a third-party service to detect bots (the link to the article explaining this can be found here: #201 (comment)) which would explain the inconsistency. For reference, I am running an Ubuntu server as a virtual machine on my laptop hitting the 403, while my host works perfectly fine. That's my main reasoning as to why I think it is the 'server' environment as Docker also has a very similar environment to servers (headless, no graphics server, etc.) |
|
man that sucks, I wish they would just let us be! lol |
|
Got 403 errors in Docker on both headless linux and headed Mac servers. Managed to get it working by running python script directly on headed Mac Mini server by setting Thanks! |
|
Have you tried installing a XFCE desktop environment on the remote Ubuntu server and configure it to run a VNC server? - Something like this https://vitux.com/ubuntu-vnc-server/ . I'm thinking that way you may be able to create impression that the ubuntu server is a machine that does have a display, and prevent the Forbidden 403 error. Could be a reasonable workaround if this works. |
I have set up a VNC server through Docker, but I unfortunately still saw the issue. I'm also seeing the issue now with a desktop so it's definitely doing more to detect the bot than just seeing if the client is using a desktop or not (I'm not sure how that detection would actually be done). |
|
I would like to add that they're definitely doing more to detect the bot. Still seems hit or miss though whether its 429 or 403. I've done several tweaks to try to slow down and randomize when things are done, but may be more to it... |
|
Maybe this has been covered before, but is there any particular reason why the mobile site has to be used for this script? Many moons ago, I had written a SWA check-in script in PHP, using the standard website URL as the target. I'm sure it's changed significantly since then, but maybe the main site is less restrictive when it comes to bots/scripts? |
|
I'm running an Ubuntu VM (via Unraid). I was getting a 403 on every attempt. The suggestion in comment #230 resolved the issue for me. |
|
@rnegrette - I'm running it directly on my Ubuntu server which will work normally. However it is hit or miss at this point in it's detection and throwing errors of 429 and 403. |
|
I'm running it directly in python on a linux server and getting 403 errors each time. |
|
I can confirm that installing a lightweight window manager (even if the logged-in session isn't using it) works. |
So, can someone try doing that in a docker container? Seems super silly, but hell, if it works, it works? I like the randomized wait times, too.. |
|
On Mac, running the application locally: AttributeError: 'Lock' object has no attribute 'is_fork_ctx' Through docker: Reason: Forbidden 403. |
|
@devtrf for the issue you are running into locally, can you make an issue or discussion for it? |
|
Ladies and Gents you won't believe it. I was able to successfully check in with v5.0 no errors and price checks successfully. Running on Docker with Synology. Try it out, let me know My observation is that SeleniumBase chrome could be the cause for headless in Docker Edit: v6.1 works! - STABLE (using it for my check ins) - successfully logged in and checked in 4 flights without any errors. Will stay on this version. v7.0 works! - unstable |
|
@dmytrokoren I'm still getting: I'm launching this via: update: |
|
v5.0 is working for me as well. |
v5.0 is the most stable as of now for most but I'm still having luck with v6.1 via login or reservation. |
and others
Version
develop tag Auto-Southwest Check-In v7.2
Browser Version
Using browser version: 121.0.6167.184
Description
I'm getting consistent 403 errors when Retrieving reservation information. I purposely limited my config down to a single record locator.. removing the account login stuff and setting check_fares to false. I updated to the latest develop tag just now as well. I ran it 4-5 times in a row and got the same result each time.
To Reproduce
Running it via docker-compose w/ config.json w/ a single record locator.
Expected Behavior
I'd expect it to schedule a checkin at minimum, and ideally successfully check in at that scheduled time.
Relevant logs and program output
my config.json { "check_fares": false, "notification_urls": "gchat://XXXXXXX", "notification_level": 1, "retrieval_interval": 6, "reservations": [ {"confirmationNumber": "XXXXXX", "firstName": "XXXXXX", "lastName": "XXXXXXXXXX"} ] } docker logs output: 2024-02-26 15:23:21 DEBUG MainProcess[log:23]: Initialized the application 2024-02-26 15:23:21 DEBUG MainProcess[main:112]: Auto-Southwest Check-In v7.2 2024-02-26 15:23:21 DEBUG MainProcess[main:113]: Called with 0 arguments 2024-02-26 15:23:21 DEBUG MainProcess[config:120]: Initializing configuration file 2024-02-26 15:23:21 DEBUG MainProcess[config:149]: Reading the configuration file 2024-02-26 15:23:21 DEBUG MainProcess[config:163]: Reading configuration from environment variables 2024-02-26 15:23:21 DEBUG MainProcess[config:58]: Setting check fares to False 2024-02-26 15:23:21 DEBUG MainProcess[config:80]: Setting notification level to <NotificationLevel.INFO: 1> 2024-02-26 15:23:21 DEBUG MainProcess[config:93]: Using 1 notification services 2024-02-26 15:23:21 DEBUG MainProcess[config:97]: Setting retrieval interval to 6 hours 2024-02-26 15:23:21 DEBUG MainProcess[config:139]: Creating configurations for 1 reservations 2024-02-26 15:23:21 DEBUG MainProcess[main:142]: Monitoring 0 accounts and 1 reservations 2024-02-26 15:23:21 DEBUG Process-1[reservation_monitor:60]: Acquiring lock... 2024-02-26 15:23:21 DEBUG Process-1[reservation_monitor:62]: Lock acquired 2024-02-26 15:23:21 DEBUG Process-1[checkin_scheduler:50]: Refreshing headers for current session 2024-02-26 15:23:21 DEBUG Process-1[webdriver:106]: Starting webdriver for current session 2024-02-26 15:23:23 DEBUG Process-1[webdriver:122]: Using browser version: 121.0.6167.184 2024-02-26 15:23:23 DEBUG Process-1[webdriver:126]: Loading Southwest Check-In page 2024-02-26 15:23:27 DEBUG Process-1[webdriver:64]: Waiting for valid headers 2024-02-26 15:23:27 DEBUG Process-1[webdriver:155]: Waiting for headers_set to be set 2024-02-26 15:23:27 DEBUG Process-1[webdriver:159]: headers_set set successfully 2024-02-26 15:23:27 DEBUG Process-1[reservation_monitor:84]: Scheduling flight check-ins for 1 reservations 2024-02-26 15:23:27 DEBUG Process-1[checkin_scheduler:76]: Retrieving reservation information 2024-02-26 15:23:41 DEBUG Process-1[utils:39]: Failed to make request after 20 attempts: Forbidden 403 2024-02-26 15:23:41 DEBUG Process-1[utils:42]: Response body: { "code": 403050700 } 2024-02-26 15:23:41 DEBUG Process-1[checkin_scheduler:82]: Failed to retrieve reservation info. Error: Forbidden 403. Exiting 2024-02-26 15:23:41 DEBUG Process-1[notification_handler:76]: Sending failed reservation retrieval notification... Failed to retrieve reservation for XXXXXXX XXXXXXXXXXX with confirmation number XXXXXX. Reason: Forbidden 403. Make sure the reservation information is correct and try again. 2024-02-26 15:23:41 DEBUG Process-1[checkin_scheduler:57]: 0 flights found under current reservation 2024-02-26 15:23:41 DEBUG Process-1[checkin_scheduler:43]: 0 total flights were found 2024-02-26 15:23:41 DEBUG Process-1[checkin_scheduler:100]: 0 new flights found 2024-02-26 15:23:41 DEBUG Process-1[checkin_scheduler:104]: Scheduling 0 flights for check-in 2024-02-26 15:23:41 DEBUG Process-1[checkin_scheduler:116]: 0 flights are currently scheduled. Removing old flights 2024-02-26 15:23:41 DEBUG Process-1[checkin_scheduler:132]: Successfully removed old flights. 0 flights are now scheduled 2024-02-26 15:23:41 DEBUG Process-1[reservation_monitor:71]: No more flights are scheduled for check-in. Exiting...Additional context
Synology DSM 7.x
Docker version 20.10.3, build 55f0773
The text was updated successfully, but these errors were encountered: