Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide Jupyterlite as an option where possible #64

Open
wragge opened this issue Apr 22, 2023 · 5 comments
Open

Provide Jupyterlite as an option where possible #64

wragge opened this issue Apr 22, 2023 · 5 comments

Comments

@wragge
Copy link
Contributor

wragge commented Apr 22, 2023

Wherever possible add Jupyterlite as another option for running notebooks. In the first instance focus on notebooks that will require minimal code changes, for example, the Getting Started notebook and the Trove API intro notebooks.

Also look at heavy-use notebooks such as Trove newspaper harvester and save newspaper images notebooks -- these will require more code changes but but make things a lot easier and quicker for users.

@wragge
Copy link
Contributor Author

wragge commented Apr 22, 2023

To minimise code changes use pyodide-http to patch requests. Eg will just need to include something like:

%pip install -q pyodide-http requests

import pyodide_http
pyodide_http.patch_all()  # Patch all libraries

Then requests, or libraries using requests can be imported and used as normal.

@wragge
Copy link
Contributor Author

wragge commented Apr 22, 2023

Note that the most recent version of pyodide-http (0.2.1) fixes a problem with Firefox in Jupyterlite. However, Pyodide currently packages an earlier version of pyodide-http (0.2.0). There doesn't seem to be any easy way of overriding this, so I'll need to wait for Pyodide to be updated to avoid problems with Firefox.

@wragge
Copy link
Contributor Author

wragge commented Apr 22, 2023

A quick investigation indicates that I'll probably have to make some changes to the trove-newspaper-harvester package. In particular, there's a problem with the way requests-cache creates new databases in the Jupyterlite filesystem. It looks like a way around this is to create the db in /tmp/. This is probably a good idea anyway.

@wragge wragge changed the title Use Jupyterlite where possible Provide Jupyterlite as an option where possible Apr 22, 2023
@wragge
Copy link
Contributor Author

wragge commented Apr 22, 2023

See also Voici as a JupyterLite based replacement for Voila.

@wragge
Copy link
Contributor Author

wragge commented Apr 18, 2024

Requests is supported now, so the code changes should be minimal. The main problem now is going to be dealing with CORS issues. CORS implementation in v3 of the Trove API is currently broken. Even when (if) that gets fixed, I won't be able to scrape from web pages etc to get extra metadata. I really don't want to have to use a CORS proxy...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant