Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Not handling 429's gracefully #70

Open
jasonwbarnett opened this issue Aug 7, 2021 · 53 comments
Open

Not handling 429's gracefully #70

jasonwbarnett opened this issue Aug 7, 2021 · 53 comments

Comments

@jasonwbarnett
Copy link

jasonwbarnett commented Aug 7, 2021 •
edited

This may not be directly applicable since I only ran into the error condition after I changed the code in a fork (lowered retry sleep), but it still may make sense to add exception handling for the situation where Southwest rate limits requests and returns a 429. Thinking that adding an additional sleep (i.e. 5 seconds) may make sense.

Response

HTTP/1.1 429 Too Many Requests

Body

{
  "code": 429999999,
  "message": "Error.",
  "messageKey": "ERROR",
  "httpStatusCode": "BAD_REQUEST",
  "requestId": "",
  "infoList": []
}

For reference: https://datatracker.ietf.org/doc/html/rfc6585#section-4
@zxlin
Copy link

zxlin commented Aug 8, 2021

Can confirm this exact behavior as well - happening from both AWS and residential IP addresses. (Thought it might've been blocking AWS IP, tried residential). This was not happening earlier in the week. Looks like something just changed very recently.

On latest master commit (6738eff)

@jasonwbarnett
Copy link
Author

jasonwbarnett commented Aug 8, 2021 •
edited

Yeah I used the bot successfully on Tuesday (August 3rd, 2021). It failed to work yesterday (August 7th, 2021).

@byalextran
Copy link

hmm. i wrote a ruby gem that's similar to this. also ran into a 429 error today when things were working a couple months ago. i tried sleeping 30 seconds between calls but still get the error. i think there's something more going on than just rate limiting.

unfortunately, i'm traveling this week and don't have much time to troubleshoot. but hopefully the hive mind will uncover something so we can all fix our code.

@mmoore99
Copy link

I have a .Net checkin program that has stopped working as well that is getting the same 429 error described above. I think it may be related to the issue discussed here

@QuiGonLeong
Copy link

Same problem here

Sorry! This reservation is not eligible for check in.
This flight is currently not ready for check-in. Please try again momentarily.
Attempting check-in...
Error.
Error.
Error.

@rbrotz69
Copy link

I had the same issue where it just showed about 20 'errors' in a row. I manually checked in and noticed a Covid 19 health acknowledgement screen you need to accept first. I wonder if that is what is screwing up the script?

@dwightmulcahy
Copy link

Yeah, noticed it isn't working now... :(

web.1 | 18-Aug 20:39:51-INFO: Attempting check-in...
web.1 | 18-Aug 20:39:51-INFO: Error.
web.1 | 18-Aug 20:39:52-INFO: Error.
web.1 | 18-Aug 20:39:52-INFO: Error.
web.1 | 18-Aug 20:39:53-INFO: Error.
web.1 | 18-Aug 20:39:54-INFO: Error.
web.1 | 18-Aug 20:39:54-INFO: Error.
web.1 | 18-Aug 20:39:55-INFO: Error.

@zxlin
Copy link

zxlin commented Aug 19, 2021

I had the same issue where it just showed about 20 'errors' in a row. I manually checked in and noticed a Covid 19 health acknowledgement screen you need to accept first. I wonder if that is what is screwing up the script?

it was definitely working this past year when there was a COVID acknowledgement already, I'm inclined to think that is irrelevant.

@jaredhughes2
Copy link

I was able to track down the problem, but haven't found a good solution. I'm hoping that others can look at it also and try to find a permanent fix.

The problem is that some headers are missing that weren't previously required. One of them is the user-agent header, which can be hard-coded so it's not a huge deal. But there are several others that are required, and I can't figure out where the values are coming from. The headers all start with ee30zvqlwf, and once I figured out that was the problem, a quick Google search led me to this project: https://github.com/redfern314/southwest-search. There are notes on a temporary fix in the readme file in the project, but it says the workaround might work for a number of days and it only worked for an hour or so for me. It looks like that project was updated with those headers over two years ago, so it looks like it's been a problem for a while (just not with the mobile service until recently).

I'm going to keep looking at it when I have time, but it would be nice if others could try to figure out where those header values are coming from as well.

@zxlin
Copy link

zxlin commented Aug 25, 2021

This also might be useful reference: https://github.com/Fffrank/southwest-alerts, they're just capturing the headers and using it again. It might be a valid strategy here too.

@byalextran byalextran mentioned this issue Aug 26, 2021
Closed
@schmeed
Copy link

schmeed commented Aug 26, 2021

I have a .Net checkin program that has stopped working as well that is getting the same 429 error described above. I think it may be related to the issue discussed here

Hello, do you have your .Net code on GH? Willing to share?

@jaredhughes2
Copy link

This also might be useful reference: https://github.com/Fffrank/southwest-alerts, they're just capturing the headers and using it again. It might be a valid strategy here too.

I hate the idea of relying on "browser" clicks to get the header values. Ideally somebody will be able to reverse engineer the API and be able to generate the necessary headers each time, but until somebody has time to try to figure that out then this can be used in the mean time. I don't know when I'll have time to do that (might be a couple weeks), but I'll look into it when I can.

@jasonwbarnett
Copy link
Author

jasonwbarnett commented Aug 26, 2021 •
edited

@jaredhughes2 I've been thinking about this and I was wondering what if we used the Selenium WebDriver to do the browser part automatically with a headless browser? It could be cool, but the additional dependency is annoying.

@rbrotz69
Copy link

rbrotz69 commented Sep 3, 2021 via email

@rschoen
Copy link

rschoen commented Sep 11, 2021

I made a temporary workaround, until we figure out something better: https://github.com/rschoen/SouthwestCheckin

Grab the headers from the browser / burp proxy / whatever, and paste them into headers.txt. Then you can run the script as normal. No promises on how often you have to refresh the headers.txt file though. Other sources indicate it's valid for some number of days.

@Jeremy-Harper
Copy link

@rschoen
I only get a response of access denied. Can you provide more instructions in your fork?

@Jeremy-Harper
Copy link

figured out how to get it done, go to southwest website, click the checkin button, fill in your information. Right click in the web browser and say inspect, go to the network tab. Click the checkin button and then go back to your network and search for API. Select southwest which is the correct request and it gets you what you need to paste in @rschoen's workaround.

@Jeremy-Harper
Copy link

Can confirm @rschoen script worked this morning

@QuiGonLeong
Copy link

@rschoen updates worked correctly. The next step is have it automatically get and update the response headers.

@jasonwbarnett
Copy link
Author

I think this could easily be done using a selenium web driver, i.e. https://stackoverflow.com/questions/62262261/how-to-get-request-headers-in-selenium

@jasonwbarnett
Copy link
Author

jasonwbarnett commented Sep 16, 2021 •
edited

This is some untested code I whipped up that shows the concept.

from seleniumwire import webdriver
from selenium.webdriver.chrome.options import Options

REQUEST_URL_WITH_HEADERS = "https://www.southwest.com/api/air-checkin/v1/air-checkin/page/air/check-in/review"

chrome_options = Options()
chrome_options.headless = True
driver = webdriver.Chrome(options=chrome_options)
driver.get("https://www.southwest.com/air/check-in/index.html")

# At this point, fill in check-in form and click submit.

# Search through requests and find the one with the headers
request_with_headers = next((x for x in driver.requests if x.url == REQUEST_URL_WITH_HEADERS), None)
if request_with_headers != None:
    for header in request_with_headers.headers:
        # Pull headers from the first request here.
        print("%s: header", header)
else:
    print("Raise error here")

driver.quit()

@jdholtz
Copy link

jdholtz commented Oct 13, 2021 •
edited

I've done some testing with a very similar script to this. From my testing, it appears that only the X-Dublriiu-E header needs to be passed (along with the X-Api-Key and X-Channel-Id) to POST the data. The header resets every 5-6 successful tries, but it doesn't seem like it resets after a certain amount of time (although more testing is definitely needed). If we can somehow figure out how that header is generated, we only need to use that one and not all of the headers.
*Note: This is all from proxying the Southwest IOS App

@djre4orm
Copy link

Thanks to this branch it works as long as you manually pull the headers. I tried letting them rerun for my return flight (3 days later) and the headers were no longer valid, so the time window must be a day or two. I wrote something leveraging what Jason posted above & using browsermob proxy to copy header data. SWA site would not respond well to any selenium driven browser. I had zero luck with firefox. Chrome worked after hex editing the webdriver.exe file
https://stackoverflow.com/questions/33225947/can-a-website-detect-when-you-are-using-selenium-with-chromedriver/41220267
The request looks valid. I can copy it as powershell code with the headers & get a valid "Wait till 24hrs" response. However with those automated headers its back to the original error at the time of check-in. Reviewing the headers, the only thing I see missing is "ee30zvqlwf-a0". For some reason with selenium that header doesn't appear. I have tried without the proxy & leaving the automate chrome window open to inspect the headers & still no a0.

@jasonwbarnett
Copy link
Author

@djre4orm nice find on the hex replacement to prevent sites from detecting the use of selenium web driver. I'm going to test this with vim on Linux and see if I can get positive results.

@h4xdaplanet h4xdaplanet mentioned this issue Oct 29, 2021
Open
@altodd
Copy link

altodd commented Nov 24, 2021

I will look into this soon, but if the headers can't be easily reversed I could see the possibility of just making this a multi-docker app and using a dockerized Android Emulator (https://android-developers.googleblog.com/2019/10/continuous-testing-with-new-android.html) to open the app, then steal the headers from that. Or I guess at that point it could just become some automation around all the app functionality, but I'm having trouble imagining anything done that way that won't get broken by a UI change. Potentially we could use adb to broadcast intents to the app. I'm just throwing out some ideas I have as an android developer, if I get some time over Thanksgiving I'll see if I can't experiment with some of those ideas.

@Jeremy-Harper
Copy link

Failed to work today, I had updated the file from the network around noon and checkin was at 4pm. It had worked previously.

@djre4orm
Copy link

djre4orm commented Dec 3, 2021

Failed to work today, I had updated the file from the network around noon and checkin was at 4pm. It had worked previously.

Might want to double check your headers, I just had it work this morning with an 11hr setup gap. If you have done anything funny with selenium or browser automation pulling headers can get messed up. My recommendation is use incognito window in that case.

Here are my headers from a working run
:authority:
:method:
:path:
:scheme:
accept:
accept-encoding:
accept-language:
authorization:
content-length:
content-type:
cookie:
ee30zvqlwf-a:
ee30zvqlwf-a0:
ee30zvqlwf-b:
ee30zvqlwf-c:
ee30zvqlwf-d:
ee30zvqlwf-f:
ee30zvqlwf-z:
origin:
referer:
sec-ch-ua:
sec-ch-ua-mobile:
sec-ch-ua-platform:
sec-fetch-dest:
sec-fetch-mode:
sec-fetch-site:
user-agent:
x-api-idtoken:
x-api-key:
x-app-id:
x-channel-id:
x-kl-ajax-request:
x-user-experience-id:

@byalextran
Copy link

i appreciate all the eyes on this.

if i understand the problem, southwest added some new request headers that need to be used when checking in. since we don't know how those headers are created, they have to be scraped. but they seem to change regularly (under four hours?) so that new request headers are needed after a certain period.

if all of that is true, i think we can take @jasonwbarnett's code and programmatically get the request headers before each auto check-in attempt.

here's an extension of his code. if you pass in the confirmation number, first name, and last name, the code will output the necessary headers into a JSON file (request_headers.json) which could then be read later when needed.

https://gist.github.com/byalextran/e38cf6321eb78a3a1f79cd282ab69314

i don't have an active confirmation number to test the solution (i will wednesday) but throwing the idea out now for dialogue in the mean time.

if someone has a confirmation number they'd be willing to let me test with beforehand, let me know via my contact form. https://www.alextran.org/contact/

@flyize
Copy link

flyize commented Dec 7, 2021

You can always book a flight, try to check in, and then cancel for a refund. They are required by law to give you a full refund within 24 hours of buying the ticket.

@MyCodeIsSchmidt
Copy link

You can always book a flight, try to check in, and then cancel for a refund. They are required by law to give you a full refund within 24 hours of buying the ticket.

This does not apply to flights booked less than a week before the flight date.

@flyize
Copy link

flyize commented Dec 7, 2021

I've never had a problem doing it with Southwest in the past. If you think it might be a problem, you can always book the more expensive, fully refundable fare. Not like price matters when you know you're going to cancel anyway.

@byalextran
Copy link

byalextran commented Dec 8, 2021 •
edited

got a confirmation number to play around with this evening. unfortunately, something's still not working.

i can dump the headers programmatically but can't check in with them (429 error). however, if i manually save the headers from chrome, i can check in.

based on my testing, it seems like these are the minimum required headers (on the POST call):

{
  "x-channel-id": "MWEB",
  "x-user-experience-id": "2e947457-6e15-4505-b230-b91c4668b071",
  "content-type": "application/json",
  "x-api-key": "l7xx44e0282659f14664b831eeba67b38a6e",
  "ee30zvqlwf-a": "0jcTP8QWfLUAcSWNY1SUECc9nS1N_yAOtKd32pWHrJ839B1YX0zkJFJVAKQ=i9MlqKsyigspmzjM1MpTPRIXQ7LSTMa0ffTHlcZmrCt0DRKiFf_16ASCHoCpgeXCAqZB6AIaZ_YtfEslleiCaKY7l9Ym1_9YT1jM8nmwieSIV=uovrv_LPYD5awg2eZ-9Is91V5cpQ2oP4SePcQ-ez31TIOt9FmTRZ3vQ61HEPNzoPVWSH601g5WEuKyEVNqphVjMzm=m0f8n3CfIigsrZhpCsV5mKq4EKQPWJqnzkUU2IPT3m8llwXbMPEBonDVD1glZkpuTc4K0AYByy=bqCMOwFKVR8IFdFkbmnVwUk-WXJHHMOVAfO_AQkQRaX4sj96_ukR78R6hdQLWrr8UbRgjtuctCBfLQj3XgLtwWIcyw_=K5IqX4i8Csr=YJC9NKtVmqKbFB69LX5ZmIRhnwt=cD0OB9-TA8rSbeBYAQT=uV2ZWt7nEV5DHQPPLRnu2uz4_bS53DLZXmykF209Eisq2Rdm-COr3ndpKHyFdVJsdRY4uy4uZpipX9kiQoQ=V-=nBHDOkKeBnjZ1KRJd7b6poLEFnT-ZtzdMtaS71VaCr_Vo4UkmUuQz30ZN4sN9AiKSPyvDCW8v2qVXhWzsMWpGD9tOXvruTPNW=_aFGbrWCmtJbfIKb8wuilEV8sGBKKBH=T1MQoWCsvgnS4A5dNtWLl=Y==2Pnk86I8dF4Yg7QAghRvsQEJsClJJQkDoMBjFHmM_PvJr=SIjD106bOIlF7KeI2lT7Q72DQE_n=2gp0r-efU2hJ0Fv=3psBOiFb5b2FDDgQtkkNpHgLPCc2LIJCrrmGqPGMYjYbjc=ciqJuqj3yMuQaFrgmFE8nmlXWaIS1_70tbZD3FbG0CfO2c1KeXvZqX9VwoLXo=fa7VzA3TyZS32z30XZ_MAKMAmIfH8c3gCrmCoB_RgdCNR=WngbheeFqL7DHZjaWqXyBqQn3dSf_P7zUvDNVNW671TAJKKsQmOF0kE7sP1e9-wb7C-WvGAgEz8HONrvk7TNLM4VlstgNb2VWz0KZvKmrOycuZs5j50a9_YhnY1HeMcIZHucdl18=va=D-OrEmZzR6IfEtAAiK7cWIRUu57m0f3F6uSAmZc0AGeZ-Dd=V5=YIcdwFhCH-QWeBMIYVmSI8hsSUJQ07sB3Yn5_Nh1PWYOTLcLQ6Gs8Uv11hwI0oFz1jzepPUXq5a0OB-qch4KWey=JD41EPW5fT9RRpIPw3kzzCDTMXHNjVEKAbT4lVicKrCWr1BLfaVB0sR9kKaITluYc_Ft4ej6KZfrHd0K034tYTNuZh6AmLId6CfWdtF3qPR7FB8MAzA=J1T3YUVAHX07sTAR6Y62W_-=DpStBLqmXByB7Ztl68JLCk6repiBGby6VmP2f5sIobEsWRYVWJh=N66EGDBW04Sm1BoEkXyV8re7fYQ=FIWUauIDHXeipUpzy8YW6fE9Tj5Zt8gjAwWgUPbm533oNa8AtMXaCPqr=e=UgnEyP8nwdBaSgAOB68GbQG2l5SzE_vjs3v0pY0fcHKTVwuU7KlIXkjDNj3hm7PY0sINlFnELc1fw7uG=HYDcZqSErphKdLpPlNMEyoLYRN-_ZVZq_r2MlUNXo6-NAa6v-j898hzhN2yorLXNNFjn6mEO5tt-GK45AiDJVL-dSXACt5hkBIA_0Gg2A-fhg-rTl6rP1mADbrOYP-JeDvqKCWQvRlkTYQ5_imaru1DOQXYOaXDbGUHeJZj1j1fma75yVbvGGqQBq9fVagiWvdgDXl8DpD==_ybNvGMp193JCI0097INpumjmcmdJuAYu6HNtj2D2T1NEiliNJt5gdcKHSmKCbUh7gIGpJ-MY6bbRKzpfjAQ52Kd2N72UJ7ksAWfyGQlsC6s7gRrJvkQUpmCXfm145djXehaXlAtZOAu8lHHSiQke6UslchMcL2_QO3J-ksBbsKDM_jter_KrLEe8spE8QOnTMLqTeQJjkaruLRZ7RABedzM5la_ajbzi3sOFUsCoDXfUY9Tyz2YJzfHrXryU=7HduXfYK=pOOOSI-a4mZL-A5lTPPRSuK8Pu_np0oH9hiL4bN6r-F46cuQ9VtB6yYiIOWR9SaE0Q1cFScKg4klMvXePY9VvllSl6RWUhp4r9urVZeS0wkjhjj7C2fsKK_mHH5SJvGO5PgvQUoJwQpQugAcnXmR0ivc-77uMhoC5krvMEzcH96vQ3pXtDSPRq3XLIeYngnA_lLUeMcj_8froUDJjO9D0klPIkYjN6W-ZwfYy1NtQgF4_lzRDTVsJXI3bBSXt1am9GvAVhUzo=5FWRn9SEvVsjvdMfVv2bFb7vv4NPcCnlZQZ4o3H0Z4HU5Q1lFwEGEy6tJPd=gYoT8-mLje4go1o9VU4j60YlyN5B3d=0=DnJvBzBwfM_nYoRT-Bs0OD4TE7iSXN8FdLOIDjTnt_0v1SFA7kDE3IZhMjI4DVMPztA-4Vz4vVZy4a_da3=0HkqEK=hU-BMhBovju_KC7O3wfuEcVGY4QdBIzYnfsw-WSpr9YqQl0BIc_sziYmJ2BeqoI8L3zgVshlUOQdU1Y39pgkZFkrQriISyV5BTmadzgeC=1W0YuTS8Eq55-mhKUK4RzA2PZEqyD5y3a_GtL6o4iUQnfcri5gcpsysMHZQGjZ8-bMGUGCDePHe8JQ44WRM5VRJJ8cal7OOjXe8NlwlQzuDNjI1yhQIgIW2l6P061_4G234jotZ7nJMa_DOy2rTsI5FsB0sJWU=g_jZmO1516NMi41ZCd7Sngy-OdNAwYUeKCnUEGS4cOee70lB3CGzcJI8wgL9geKDG=1ChqEdiAGrjbiW39R5iQNS8irQvncLdnEkERnBgy4RZUzN7L54vHhzo06ICX6SCNup9GkrvamE0kSqeJsuVs7dfff_lfEymlOQpZzvyXSIX0qaWt1ImtZrJFl_HUzS-vl_cPvugz2rCCQnSndR55vINJRMHdBHZKgRBHwTXq2qn32UyfgCMzqjpynwOeL3MpZQPDWDzcY-KuEhLKqXzDp1wv4qkuOtc1SMPL3KliljGQDzKmf8PHavh9ZsL4-NJlbkZCBmTSYdk168o3kgksK2PLQ0sDdI=IQgwoAW1S3rcXQjfnWUH6bOf=YLKn07kE1qIZtrM81BJmbnbyVm-tFr48fAogd3RgBpED1w4W0KORebMhR8wM6uTErd7HawySVsnDYfsz=8G3INPl2Adom3P3irSdz8SSLRJ6geU50f63sajeyoRGKcjlMth7htm8auR124-nsv7ndrNPSreOC==ey9liWhOE0s=bDP_ORp3DJh62PHQdRLj_3wYMImtZKhsbfGmzhg-qsQjTC=zMVDyRgdbDNGdcVksC_8_eLudmWB6qj0F2V-0YPCqTt5q9wF8WetUGkqbgLLOyZag-78Ap8YtwfhdCI3K9tUsGCeWs48Bd2s5JoOZmatfsiIqP5wr4y9jXC8qKqt-GTDsqlOEwk1aquig0z3R6kY6nSJerj7bWGlXIBtR-XEbej2GcnevftohL7IliU0ZPg4Yl63FCrDsa-8GORnOsyAkf7HvwG2fd-QuOVEmaiG9qcIyjDlszCEDYGdj-Xa8mSm4bSBmp053pBeANZCa_s-k77I-3CKN29wd7ldoWnP3I9=fnWDv5Mqf_KymjiC-3D8-OfrMn9_V4kw9fHLXd1JtB3DumYlEq9VEBS=1CoHSBS0LjUtWA581KXqVv3ysB6XECpb=Df8yDn6gMI_329Df3mwX4gcyypV01vXMrG=BeDuzCaovFWvYrbYEr5PFFyvwP7Hh_2J509ymOB6Pz7DRtQwhX-w5lCjRJlpg5nG6yt-m33m-Vw9XUT2Cey2tqwLG0prsPAW_QwAbpIk21rGraB3_vB_TDJnaPqwHfjwCnZiFLZ=3iLQtCfTKrQkXT6o6L5BgstrMSuXmQidOrhfkqr1YB05_9uks8lo-ufSVfDNuqpolOEW6Gb_XM1N5mlEL5kJUSB2YmV3IL2Z7O3A_rPz5bVulGYg4qNS=ZiISPSvIWUDFKMa9lMzDmqVy-z46kjW=5ustsm2Takmrd4i05JE=C4dEb3BfYFMJtwMzBdaBgAwaiABAig-KDyp3CCN73nHjApC_1Pq33kw8B17Ldz_3y1qccz_5yhM5L5hJDlfLqRst-5ToT0UCW9ja8er_WD4CbFMAU1zmC7gt0sM6K_mW48K6sg7SdFnyNq=T7EGyoBz9OyKfEd9cz8n6FXRwc=00l-0vFXPOHenamuEX06TRd5__bd2uNuZQynNwXCgqP0a9AymT=yBPoPFY_Vwcpe3Pd-r5hU2q7s42SLDyR9RfBgEaAgt6m4_7-o_bqjsKnfEaEKNw7bZnD7qiy22rJF=uiJPA4clezbn=TAi_7RZnbkCC5yISLgPFAIMZzZAZs9ig9wIPyW3IGmmVcA=EFWklbZEzN3v=Z7bB2bRjz607vCWS-HtjpBBCWeROkRLRc_5veJFoIQX=WEPNWZK=jy73vv6uY_N5qqU5pWVMpUBlRyRs1roEjdUw216=BE-neCB1gB1f-22Mn=9XqvWsHaK9C0a6SHCFwQvi=C3o_GIIMw9sGu==DP3b4jmN-bnESQHIA5-efphnTwWWTsDqiW-J=nc6VOh5cv3M2Bs23syylLgWj0EAv_lt-E-WlT41WREuzgTzTrlXwBzArOHanR3PXroiK2vQiSpL=0gp8UQUl91NXQORbUiEpSDPs0hI_XOz0Ye6ezs-HHNCV9jJvemwNfqsMUpIrE7LHTt5Vn4BwAqH18EJuKzZVfHGXZnBHhBtysajs2kGJOTQybbzGqF_-PdRFGJhlR5GXF5zow26QdCjOfg94g9qYEln4IcyYArnbd_b_OWK-YZUyB6CIV5nO1ssE3k8zQ=eAWbkzGR0CTtFNdBUIIZJ9CrUHG_W6oR=yUMwAFwHf-gWgAcnTGz0WL5zutsH-UNeU0UMk_K_HGfckwmtGhOhtDhCD4y-3j-KC5A-PmjVtAe2i9PSk0dzpWfsDGgJWvfMnJJvwdS0PUlfqMcnaE21AYhd3Dqh9aBSOJmIl_hrvpgNdZH0yfrMrHO-XOfCRPhIuUL1c0oHPUT--180n3nsefy5GqW8cfMaJiBaedW-IRPkWZg0q64vG_CHYdW_ckrO=RdVIB2Tk9vXyAqG1wyvrYbw=O90IGlJpptEN0biIdTh5jaBX-hjBtWVpRj1-zAZypl9YqSSIgM2a3s0dPyMGPcZAvE=Il8OY0RwXehtIl5CJ1EWQHU1JFCrC_pRBN4UQpGf-qLj=ZZGe0NlwuFVKGyfLDABqX8VoANE97dGIjle=qzXHj03V-5ZhIOV3b3iGMHccrOLh15blR_8iE=Pfusph47VrdtDdVDoW02lu3H7vAi5lUEIoGT7Kd12QEPM0IqNPutPjXm7nCoUCLe6NF_MvAhS0FiVr_eL2r_=utPeWphAEzQV1u00KYhNJ5KUMYjsBb=JW2_GwFYr_5-7ikrk7Iz9ymUI4Yr_zJ3=T2nWG-rQ8QZf0Z_bp-N83WbBiJ9wEW__2YEuAzguleGDLt0IZd1sA23ykdzv-081i7aWjmUpfRgRPddKXwCYRiaSbJtzuNhtZbFqFm9zILrjU8AoXTeqomECb06FRlgDXvReb0FJMhBrB7J71Vw1YmQdkTndKQ2Tbg6M6RG=BSDOGpVquvnR3IuKbGiD2rYcHv=0Xo2yu_FVLkE3hE4T-lSKW_vaYusNHy1REa7qHwofCq-Hh1QOdfUZS9SEN3I1=D-y5upb_aC5kLB_1RAkcfXteTqfMFvL-09l=Hd2FVQfFj5JHo4G6Lg8sSt-j-Gc53rMP1WS8NW=pJqJuJ0q8QbjlQhuawBj0aEdcPhXhs9ZWXgl5hl9UMYin_GVm3HjsTDNPQbujAF1lV3t7=Qmo1NqNCZ2ky8gn1Ob3LAyj97aBKWnytB8tRYXX7PaLKiuIodIMlJWO5IThKDp-6tsIZo-PDHLm80FCpyL3s=LVA0UkENMUVSIpA1cjcwDbO48tU_cJrSOjBsbTseeDDh58Pek2jWSmkNCjkRL6ha6rZB8zAZKdisqj5r2QokX37u1l-1eoYP1SKJRl1gN1MhPDOZd9-XkjT66wXisaZd3MpeHiTv1LI-h9b=ztJtrk=9LZ5Ir6QXjki2BILpwDhHoUpFIaw-R1npK09DU9FD-io_oLU0Gama6srMkmEV8P11QtQt9vtVBjDckOYu8mN=J18dGoF2YAv07PIe5SI2MSwTyo9m0gz5dnI00SMDMOkfIOgjC65Il_SElgudF9O1XncJ-LHqKk58UYMj7GBTU",
  "ee30zvqlwf-z": "q",
  "ee30zvqlwf-b": "mnk054",
  "ee30zvqlwf-d": "ABaChIjBDKGNgUGAQZIQhISi0eIApJmBDgDQiciH205akf____-jCnTyACJzXjHjw3jQzDCeQcbIIuw",
  "ee30zvqlwf-c": "AOAugJd9AQAAyWpQ4p1MgPIjRX-9FZic2VdmlR0kJ6TaQRU7QNCJyIfbTlqR",
  "ee30zvqlwf-f": "AwKagpd9AQAAOyeNxr2iG9LVC3nztxjqTGu9kBXyhiGzD8_v8ctAF9mDiMpSAQgGcPWuctP0wH8AAOfvAAAAAA=="
}

i've tried checking in with those exact headers pulled programmatically and manually (same keys, different values). but the manual one is the only one that works. so something about the values is different that's causing the check-in to fail.

right now just at a loss for what's different.

has anyone gotten the check in to work using headers pulled from @jasonwbarnett's code?

i'm wondering if all the reported successful tries in this thread are done from manually header pulls?

EDIT 1: actually, re-reading this thread more closely, i see #70 (comment) already came to this same conclusion. 🤦🏻‍♂️

EDIT 2: the programmatic headers work after hex editing chromedriver! will now see if i can integrate this into my gem https://github.com/byalextran/southwest-checkin. it'll be a frankenstein job though since my code is in ruby, the programmatic header dump is in python, and you'll have to manually hex edit chromedriver. 😅

EDIT 3: programmatic headers don't work if selenium is run in headless mode. not sure how this will affect integrating everything into a command line tool. 🤔

EDIT 4: headless mode works if you set a user agent that doesn't have the word "Headless" in it. (apparently when you set headless mode, the user agent then includes "HeadlessChrome."

alright, headed to bed but hopeful that a solution is on the horizon.

@byalextran
Copy link

making some progress. i created a new repo for code that will output the proper headers needed for any check-in scripts.

https://github.com/byalextran/southwest-headers

will try to update my project later this weekend to support it and get a fully automated solution working again.

kinda what i'm thinking is have that script run in a cronjob every day (doesn't appear to need a valid confirmation number to generate working headers) and then update my project to grab the header file before checking in.

it would be helpful if we could collectively figure out how long a set of headers is valid before needing to get new ones.

@byalextran
Copy link

byalextran commented Dec 12, 2021 •
edited

Happy to report my auto check-in script is working again!

https://github.com/byalextran/southwest-checkin

Thanks to everyone in this thread for contributing to the solution.

@swtools0
Copy link

@byalextran your headers work is awesome! I've implemented it into my AWS Lambda solution here: https://github.com/sw-tools/checkin-service

@jdholtz
Copy link

jdholtz commented Dec 18, 2021

@byalextran great job figuring out how to get the headers! One optimization you can do is using an undetected-chromedriver. This takes away the need to manually install the chromedriver and hex edit it.

@byalextran
Copy link

@jdholtz oh nice, thanks for that tip. i'll incorporate it on the next update!

@bjhiltbrand
Copy link

@byalextran I made a PR for the move to undetected-chromedriver. I tested it and it seems to work great.

@hildebrau
Copy link

@byalextran do you have or could you wrap a docker container around your sweet solutions? I'm impressed by your header work!

@byalextran
Copy link

@byalextran do you have or could you wrap a docker container around your sweet solutions? I'm impressed by your header work!

I do want to explore a docker solution eventually. Just not a priority right now since I have a working solution and all my free time lately has been dedicated to a new pickleball addiction. 😅😜

But see if this could work for you.

byalextran/southwest-headers#3

@hildebrau
Copy link

But see if this could work for you.

byalextran/southwest-headers#3

Brilliant! Thanks for pointing that out!

@djre4orm
Copy link

Happy to report my auto check-in script is working again!

https://github.com/byalextran/southwest-checkin

Thanks to everyone in this thread for contributing to the solution.

Set this up and pretty sure I had success this AM. Thank you for your work! I had to add the autoluv dir to my environment variables before the command would run, maybe it's just how I installed my ruby.

Side question: is there any way to setup a shell output for the result instead of email? I tried looking in the log dir you mentioned autoluv-0.3.1/logs/ but no such directory

@h4xdaplanet h4xdaplanet mentioned this issue Apr 27, 2022
Open
@byalextran byalextran mentioned this issue Jul 31, 2022
Closed
@BigSammich
Copy link

has anyone tried this python version recently and had success? I've been running it today and am getting a BAD_REQUEST.
{ "code": 429999999, "message": "Error.", "messageKey": "ERROR", "httpStatusCode": "BAD_REQUEST", "requestId": "", "infoList": [] }

I'm generating headers using
https://github.com/byalextran/southwest-headers, but it appears that something changed again because when I manually go through the process using https://mobile.southwest.com/check-in I see that a ee30zvqlwf-a0 header now exists on the POST
image

when I print out all the headers that https://github.com/byalextran/southwest-headers processes, ee30zvqlwf-a0 is not one of them, so wondering if it could be a Selenium issue. As expected, just copying and pasting a previous value does not work as I believe all the ee30zvqlwf header values are generated together.

@h4xdaplanet
Copy link

h4xdaplanet commented Nov 12, 2022 via email

@BigSammich
Copy link

Yes, I did the below chromedriver modification
image

these are the request headers I'm currently getting
image

If no one else is having problems anymore, then it must be something local to my machine.

@byalextran
Copy link

@BigSammich southwest did something to detect my southwest-headers script. i finally got around to troubleshooting and just pushed a fix.

follow the instructions here:

byalextran/southwest-checkin#13 (comment)

@3052
Copy link

3052 commented Mar 21, 2024 •
edited

from my own checking, it appears the protection being used here is called apiguard3, and is now being used by other sites such as https://peacocktv.com

if anyone is able to figure this out without requiring a headless browser, I am willing to pay 1000 USD. my contact info on my user page.

@3052 3052 mentioned this issue Mar 21, 2024
Closed
@hildebrau hildebrau mentioned this issue Mar 21, 2024
Open
@FrankGasparovic
Copy link

@3052 Can you share more details about apiguard3? I'm not able to find much information on the internet on it.

@3052
Copy link

3052 commented Mar 22, 2024

@FrankGasparovic I think its actually called ShapeSecurity

@dwightmulcahy
Copy link

dwightmulcahy commented Mar 22, 2024 via email

@bllfr0g
Copy link
Collaborator

bllfr0g commented Mar 22, 2024 via email

@bjhiltbrand
Copy link

I used to use this script, and it worked great when I used it. But recently, I have been using this docker container:
https://github.com/jdholtz/auto-southwest-check-in
It was easier to setup and cleaner and has been running like a champ. Take a look over there.

@3052
Copy link

3052 commented Mar 23, 2024 •
edited

@bjhiltbrand that repo uses a browser:

https://github.com/jdholtz/auto-southwest-check-in/blob/master/requirements.txt#L4

I am looking for non browser solution if possible

@hildebrau hildebrau mentioned this issue Mar 29, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests