Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SSL Error When downloading dataset #1233

Open
junpuf opened this issue Feb 15, 2024 · 3 comments
Open

SSL Error When downloading dataset #1233

junpuf opened this issue Feb 15, 2024 · 3 comments

Comments

@junpuf
Copy link

junpuf commented Feb 15, 2024 •
edited

Your issue may already be reported!
Please search on the issue tracker before creating one.

Context

  • Pytorch version: 2.2.0
  • Operating System and version: Ubuntu 20.04

Your Environment

Expected Behavior

Passing

Current Behavior

Failing for SSL error when downloading dataset

Possible Solution

Steps to Reproduce

  1. download this repo
  2. bash run_python_examples.sh 'gcn,gat'
    ...
    or just run
import requests
requests.get('https://linqs-data.soe.ucsc.edu/public/lbc/cora.tgz', stream=True)

Failure Logs [if any]

bash run_python_examples.sh 'gcn,gat'
using cuda
Starting gcn
Running example: gcn
Using cuda device
Downloading dataset...
Traceback (most recent call last):
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 715, in urlopen
    httplib_response = self._make_request(
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 404, in _make_request
    self._validate_conn(conn)
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1058, in _validate_conn
    conn.connect()
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/connection.py", line 419, in connect
    self.sock = ssl_wrap_socket(
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/ssl.py", line 513, in wrap_socket
    return self.sslsocket_class._create(
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/ssl.py", line 1104, in _create
    self.do_handshake()
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/ssl.py", line 1375, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 799, in urlopen
    retries = retries.increment(
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='linqs-data.soe.ucsc.edu', port=443): Max retries exceeded with url: /public/lbc/cora.tgz (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/examples/gcn/main.py", line 244, in <module>
    with requests.get(cora_url, stream=True) as tgz_file:
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/requests/adapters.py", line 517, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='linqs-data.soe.ucsc.edu', port=443): Max retries exceeded with url: /public/lbc/cora.tgz (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)')))
graph convolutional network failed
Finished gcn, status 0
Starting gat
Running example: gat
Using cuda device
Downloading dataset...
Traceback (most recent call last):
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 715, in urlopen
    httplib_response = self._make_request(
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 404, in _make_request
    self._validate_conn(conn)
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1058, in _validate_conn
    conn.connect()
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/connection.py", line 419, in connect
    self.sock = ssl_wrap_socket(
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/ssl.py", line 513, in wrap_socket
    return self.sslsocket_class._create(
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/ssl.py", line 1104, in _create
    self.do_handshake()
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/ssl.py", line 1375, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 799, in urlopen
    retries = retries.increment(
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='linqs-data.soe.ucsc.edu', port=443): Max retries exceeded with url: /public/lbc/cora.tgz (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/examples/gat/main.py", line 341, in <module>
    with requests.get(cora_url, stream=True) as tgz_file:
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/requests/adapters.py", line 517, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='linqs-data.soe.ucsc.edu', port=443): Max retries exceeded with url: /public/lbc/cora.tgz (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)')))
graph attention network failed
Finished gat, status 0
Some examples failed:

graph convolutional network failed
graph attention network failed

@ebrahimpichka for Graph Attention Network example
@JoseLuisC99 for Graph Convolutional Network example
@ebrahimpichka
Copy link
Contributor

Hi, I'm checking them.

@ebrahimpichka
Copy link
Contributor

I can just bypass the SSL verification by passing verify=False into the get request, for both gat and gcn.
But I'm not sure if it causes any security issues or not. @msaroufim

    # Load the dataset
    cora_url = 'https://linqs-data.soe.ucsc.edu/public/lbc/cora.tgz'
    path = './cora'

    if os.path.isfile(os.path.join(path, 'cora.content')) and os.path.isfile(os.path.join(path, 'cora.cites')):
        print('Dataset already downloaded...')
    else:
        print('Downloading dataset...')
        with requests.get(cora_url, stream=True, verify=False) as tgz_file:
            with tarfile.open(fileobj=tgz_file.raw, mode='r:gz') as tgz_object:
                tgz_object.extractall()

@msaroufim
Copy link
Member

@ebrahimpichka so because this works in CI with no issues what we can do is add an option to argparser to pass in an no-ssl flag which would disable verification. if you wanna send me a PR for that i can merge easily

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@msaroufim @junpuf @ebrahimpichka