Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to disable talking to the internet on import dace #1514

Open
havogt opened this issue Feb 1, 2024 · 3 comments
Open

Add option to disable talking to the internet on import dace #1514

havogt opened this issue Feb 1, 2024 · 3 comments
Assignees
Labels
enhancement New feature or request visualization

Comments

@havogt
Copy link

havogt commented Feb 1, 2024

Describe the bug
If dace is imported in a jupyter notebook it tries to talk to the internet.

if isnotebook():

Recently GT4Py added testing jupyter notebooks with Github actions which fail frequently in this check, probably because of some network instabilities in the runners.

I would propose to add, at least, the option to disable trying to access the internet.

@tbennun
Copy link
Collaborator

tbennun commented Feb 1, 2024

It should work offline too (it's why we catch the exception in _connected) but it looks like the code relies on material fonts from online. @phschaad any way to make it strictly offline?

We can also add a config entry for DACE_jupyter_alwaysoffline=1 if connections work but are unstable

@phschaad
Copy link
Collaborator

phschaad commented Feb 7, 2024

This is a good suggestion, and should be possible.
With a recent fix PR the number of strict online dependencies has dropped to 0, so it is possible to create a fully offline version. The only reason that the online component is still there is to always use the most recent visualization component version if it has been updated in the meantime.
I will add a config as suggested to turn off the online check.
There is still one problem that needs to be addressed (and is on my radar), though - and that is that currently the offline visualization component must be manually built first. Simply cloning dace recursively does not contain the compiled assets. I am looking into a solution for this.

@phschaad phschaad self-assigned this Feb 7, 2024
@phschaad phschaad added enhancement New feature or request visualization labels Feb 7, 2024
@havogt
Copy link
Author

havogt commented Feb 29, 2024

The problem triggered again in our CI:

 File ~/work/gt4py/gt4py/.tox/notebooks-py310/lib/python3.10/site-packages/dace/__init__.py:23
     20 from .symbolic import symbol
     22 # Run Jupyter notebook code
---> 23 from .jupyter import *
     25 # Import hooks from config last (as it may load classes from within dace)
     26 hooks._install_hooks_from_config()

File ~/work/gt4py/gt4py/.tox/notebooks-py310/lib/python3.10/site-packages/dace/jupyter.py:71
     69 # Code that runs on "import dace"
     70 if isnotebook():
---> 71     enable()

File ~/work/gt4py/gt4py/.tox/notebooks-py310/lib/python3.10/site-packages/dace/jupyter.py:66, in enable()
     64 def enable():
     65     from IPython.display import display, HTML
---> 66     display(HTML(preamble()))

File ~/work/gt4py/gt4py/.tox/notebooks-py310/lib/python3.10/site-packages/dace/jupyter.py:42, in preamble()
     39 result += '<link href="https://fonts.googleapis.com/icon?family=Material+Icons" rel="stylesheet">'
     41 # Try to load dependencies from online sources
---> 42 if _connected():
     43     for dep in sdfv_js_deps:
     44         result += '<script src="https://spcl.github.io/dace/webclient2/dist/%s"></script>\n' % dep

File ~/work/gt4py/gt4py/.tox/notebooks-py310/lib/python3.10/site-packages/dace/jupyter.py:11, in _connected()
      9 def _connected():
     10     try:
---> 11         urllib.request.urlopen('
https://spcl.github.io/dace/webclient2/dist/sdfv.js
', timeout=1)
     12         return True
     13     except urllib.error.URLError:

File /opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/urllib/request.py:216, in urlopen(url, data, timeout, cafile, capath, cadefault, context)
    214 else:
    215     opener = _opener
--> 216 return opener.open(url, data, timeout)

File /opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/urllib/request.py:519, in OpenerDirector.open(self, fullurl, data, timeout)
    516     req = meth(req)
    518 sys.audit('urllib.Request', req.full_url, req.data, req.headers, req.get_method())
--> 519 response = self._open(req, data)
    521 # post-process response
    522 meth_name = protocol+"_response"

File /opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/urllib/request.py:536, in OpenerDirector._open(self, req, data)
    533     return result
    535 protocol = req.type
--> 536 result = self._call_chain(self.handle_open, protocol, protocol +
    537                           '_open', req)
    538 if result:
    539     return result

File /opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/urllib/request.py:496, in OpenerDirector._call_chain(self, chain, kind, meth_name, *args)
    494 for handler in handlers:
    495     func = getattr(handler, meth_name)
--> 496     result = func(*args)
    497     if result is not None:
    498         return result

File /opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/urllib/request.py:1391, in HTTPSHandler.https_open(self, req)
   1390 def https_open(self, req):
-> 1391     return self.do_open(http.client.HTTPSConnection, req,
   1392         context=self._context, check_hostname=self._check_hostname)

File /opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/urllib/request.py:1352, in AbstractHTTPHandler.do_open(self, http_class, req, **http_conn_args)
   1350     except OSError as err: # timeout error
   1351         raise URLError(err)
-> 1352     r = h.getresponse()
   1353 except:
   1354     h.close()

File /opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/http/client.py:1375, in HTTPConnection.getresponse(self)
   1373 try:
   1374     try:
-> 1375         response.begin()
   1376     except ConnectionError:
   1377         self.close()

File /opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/http/client.py:318, in HTTPResponse.begin(self)
    316 # read until we get a non-100 response
    317 while True:
--> 318     version, status, reason = self._read_status()
    319     if status != CONTINUE:
    320         break

File /opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/http/client.py:279, in HTTPResponse._read_status(self)
    278 def _read_status(self):
--> 279     line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
    280     if len(line) > _MAXLINE:
    281         raise LineTooLong("status line")

File /opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/socket.py:705, in SocketIO.readinto(self, b)
    703 while True:
    704     try:
--> 705         return self._sock.recv_into(b)
    706     except timeout:
    707         self._timeout_occurred = True

File /opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/ssl.py:1307, in SSLSocket.recv_into(self, buffer, nbytes, flags)
   1303     if flags != 0:
   1304         raise ValueError(
   1305           "non-zero flags not allowed in calls to recv_into() on %s" %
   1306           self.__class__)
-> 1307     return self.read(nbytes, buffer)
   1308 else:
   1309     return super().recv_into(buffer, nbytes, flags)

File /opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/ssl.py:1163, in SSLSocket.read(self, len, buffer)
   1161 try:
   1162     if buffer is not None:
-> 1163         return self._sslobj.read(len, buffer)
   1164     else:
   1165         return self._sslobj.read(len)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request visualization
Projects
None yet
Development

No branches or pull requests

3 participants