New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
https GET request fails with "handshake failure" #2022
Comments
Sadly, this is unrelated to the issue you identified, and entirely down to the crappy OpenSSL that OS X ships with by default. Version 0.9.8y has some real problems with performing SSL handshakes, and some servers don't tolerate it well. Using Python 3 on my OS X box (therefore using a newer OpenSSL) reveals that there's no problem. You have two options:
|
Ah, looks like I was following a red herring then - I don't plan on deploying anything on OSX anyhow. Looks like I'll move my testing to a linux virtualbox. Apologies for this long-winded issue! |
No need to apologise, asking that question was the right thing to do: it's bizarrely specific knowledge to know that OS X has this problem. =) |
Ok, this is a bummer. I created an Ubuntu 14.04 server 32bit Virtualbox image via Vagrant and this is all still happening except for the SSLv2 case, where it fails because the protocol isn't included in the OpenSSL version in Ubuntu 14.04 (by design, I believe - SSLv2 is old and outdated). Versions: EDIT: forgot the OpenSSL version... python -c "import ssl; print ssl.OPENSSL_VERSION" TLSv1:
SSLv2:
SSLv23:
Perhaps this is a cipher list issue then? Or is the OpenSSL version used here still problematic? |
I am absolutely willing to put in some time to help debug this if necessary... provided you guys give me some direction. |
VM is downloading. I can't reproduce this on ArchLinux. |
@t-8ch Thanks for taking a look at this, I'm a bit confused. OpenSSL makes my life really hard =( |
@t-8ch I haven't installed PyOpenSSL if that's what you're asking? I would have assumed (perhaps incorrectly) that |
@jaddison It mostly does. Unfortunately, Python 2.7s standard library sucks hard and doesn't support some features, such as SNI. I wonder if this is SNI... |
@jaddison There are two different codepaths behind the scenes. You shouldn't have to care about those, but it helps to know when debugging. However I can now reproduce this on ubuntu. But only o Py2. On Py3 everything is fine. |
It bothers me that an absence of SNI fails in multiple different ways depending on the server in question. |
I did notice this change between OpenSSL 1.0.1f and 1.0.1g (https://www.openssl.org/news/openssl-1.0.1-notes.html):
EDIT: Ahh, nevermind - the bug shouldn't vary between Py 2 and 3, I'd think. |
@jaddison To test whether this is SNI, you'll need to install the SNI requirements for Python 2. |
@Lukasa was right. Compare: $ openssl s_client -connect docs.apitools.com:443
CONNECTED(00000003)
139846853338768:error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure:s23_clnt.c:762:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 517 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
---
$ openssl s_client -connect docs.apitools.com:443 -servername docs.apitools.com
... happy handshake here |
To elaborate: The second command enables the SNI functionality of You can a) switch to python3 b) install extra dependencies. |
Thanks for the quick feedback. Seeing as there is no bug, I'll close this... again. |
Hey, thank you guys !! I installed python3 on my mac and boom, it works. |
Just want to chime in and say that I experienced this issue on OS X 10.9.5, Python 2.7.7 and OpenSSL 0.9.8zc. I was able to fix my handshaking issue by:
|
Thanks, @Microserf. I'm pretty much running the same specs (10.9.5, Python 2.7.6 installed via Homebrew but compiled with system provided OpenSSL 0.9.8zg) and this was my entire process for getting
Install
And we're good to go: """
This may or may not be needed. See:
https://urllib3.readthedocs.org/en/latest/security.html#openssl-pyopenssl
"""
# from urllib3.contrib import pyopenssl
# pyopenssl.inject_into_urllib3()
import requests
# r = requests.get(...) |
@markstrefford 's solution also worked for me. |
Just a heads up for anyone using OpenSSL 1.1: |
Hey guys! I'm having the same issue with the following server
Any ideas? @Lukasa, I see a few issues with the Certificate, but seems like it shouldn't be that bad: https://sslanalyzer.comodoca.com/?url=34.200.105.231 |
The certificate won't usually cause this problem: this problem is caused by the server hanging up on us, so usually it's the result of a cipher suite mismatch. In this case, that's exactly what's going on as you can see here. This is a server that, frankly, should never be exposed to the open internet. There are no secure methods of communicating with this server: none, zero. This is why the handshake fails: Requests only accepts modern cipher suites, and there are no modern cipher suites available to this server. The best option is If this server is yours, please upgrade it to a better TLS implementation or change the settings. Otherwise, my first bit of advice is to reconsider ever speaking to this server. If you must, then you can use the code here, but I strongly recommend that you put pressure on the server operator to fix this mess. |
@Lukasa -- thanks for working through this with everyone! Ive read through and tried most of this IssueWhen running script on Windows it all works.
Im not convinced it is not the server itself, but would appreciate any additional help to confirm and/or pop me out of this rabbit hole. Would be a huge win to get it to work. OSX specifics:
Attempts made
I am not 100% sure that installing against the openssl actually did anything, as it seemed to act the same as installing without (such as, speed and messaging all appeared the same) As directed in another thread (above) connecting directly via openSSL
|
Uh...OpenSSL is technically fine, but that OpenSSL negotiated no cipher (that is, it appears to have negotiated |
@Lukasa Its not exposed on the internet, is there some command line probe that I could fire off that could provide adequate insight for you? |
You could try cipherscan. |
@Lukasa got it installed ... its acting wonky (no output, watching it) ... will post back if I come up with anything that could be passed along. Thanks for the guidance! |
@Lukasa thanks so much for your help - never actually got cipherscan working - but corrected our issues. It had nothing to do with any of this, and was a silly IP mismatch across our environments ... lessons learned! thank you ... |
No problem at all, glad you got it sorted! |
streamlink -l debug httpstream://https://www.arconaitv.us/stream.php?id=43 worst tried but no luck |
atlast got it working tvplayer on local pc . i installed tinyproxy in my local pc but in vps httpproxy xxxx not working . |
Hi @maanich, this doesn’t appear to be directly related to this issue, or to be a defect report for Requests which is what this issue tracker is reserved for. If you have questions about system configuration, those will be best addressed on a platform like StackOverflow. Thanks! |
streamlink --https-proxy "http://8xxxx:8000/" --tvplayer-email mxcxxcx@gmail.com --tvplayer-password vcvdf3 --http-no-ssl-verify https://tvplayer.com/watch/itv best --player-no-close --stdout | /var/tmp/youtube/ffmpeg -y -i pipe:0 -vcodec copy -acodec copy -flags -global_header -hls_flags delete_segments -hls_time 10 -hls_list_size 6 /mnt/hls/arc.m3u8 advice please n what proxy server is good for streamlink if any |
Related to #1083, perhaps. Standard
requests.get()
for this particular site/pagehttps://docs.apitools.com/2014/04/24/a-small-router-for-openresty.html
results in:Using
request-toolbelt
'sSSLAdapter
to try various ssl versions, they all fail, it would seem... see following tracebacks.TLSv1:
SSLv3:
SSLv2:
Note the last one gives a
Connection reset by peer
error, which differs from the others, but I'm pretty sure SSLv2 isn't supported by the server anyhow.For fun, I tried to pass through some more appropriate headers through on the last request as well:
No dice there either. Here's what the HTTPS connection info in Chrome on Mac looks like:
I'm not positive, but some googling indicates it's likely a cipher list issue, which is more urllib3, I think?
I tried to modify
DEFAULT_CIPHER_LIST
inpyopenssl
, but started running into import errors. At this point it seemed like things were just broken, and there wasn't really a proper way to approach fixing this yet.Version information:
OSX Mavericks
Python 2.7.5
OpenSSL 0.9.8y 5 Feb 2013 - (from
python -c "import ssl; print ssl.OPENSSL_VERSION"
)requests 2.2.1
requests-toolbelt 0.2.0
urllib3 1.8
The text was updated successfully, but these errors were encountered: